Tracking and Evaluating OKRs
Two weeks before the end of the quarter, it’s time to grade your OKRs, and plan for the next cycle. After all, you want to hit the ground running on day one of Q2, right?
There are two common systems for managing OKRs: Confidence ratings and grading. Each has its benefits and downsides. We’ll start with confidence ratings, the system I’ve outlined in my book, Radical Focus. Confidence ratings are a simple system best used by startups and smaller team, or teams at the beginning of OKR adoption. When you decide on your objective and three KRs, you set a difficult number you have a 50% confidence in achieving. This is typically noted by a 5/10 rating on the status four square.
In your Monday commitment meeting everyone reports on if and how their confidence levels have changed. This is not a science, it’s an art. You do not want your folks wasting time trying to track down every bit of data to give a perfect answer, you just want to make sure efforts are directionally correct. The first few weeks of OKRs, it’s pretty hard to know if you are making progress or not on achieving your Key results. But somewhere in week 3 or 4, it becomes very clear if you are getting closer or slipping. Each team leader (or team member, if a very small company) will start to adjust the confidence rating as they begin to feel confident.
Then the confidence rating will start to swing wildly up and down as progress or setback show up. Eventually around 2 months the confidence levels settle into the likely outcome. By two weeks from the end of the quarter you can usually call the OKRs. If they were truly hard goals, the kind you only have a 50/50 chance of making, there is no miracle that can occur in the last two weeks to change the results. The sooner you can call the results, the sooner you can make plans for the next quarter and start your next cycle.
The advantage of this approach is two fold. First, the team doesn’t forget about OKRs because they have to be constantly updating the confidence level. Because the confidence level is a gut check, its quick and painless, key for getting a young company in the habit of tracking success. The second advantage is this approach prompts key conversations. If confidence drops, other leads can question why it is happening and brainstorm ways to correct the drop. OKRs are set and shared by the team, any team member’s struggle is a danger to the entire company. A leader should feel comfortable bringing a loss of confidence to the leadership team and know that he’ll have help.
At two weeks before the end of the quarter you mark your confidence as 10 or 0. Success is making two of the three key results. This style of grading leads to double down on the possible goals and abandoning effort on goals that are clearly out of range. The benefit of this is to stop people spinning their wheels on the impossible and focus on what can be done. However, the down side is some people will sandbag by setting one impossible goal, one hard goal and one easy goal. It’s the job of the manager to keep an eye out for this.
The second approach to OKRs ratings is the grading approach. Google is the most famous for using the grading approach. At the end of the quarter, the team and individual grade their results with data collected. 0.0 means the result was a failure, and 1.0 means the result was a complete success. Most results should land in 0.6-07. From the Google official site on using OKRs, ReWork:
“The sweet spot for OKRs is somewhere in the 60-70% range. Scoring lower may mean the organization is not achieving enough of what it could be. Scoring higher may mean the aspirational goals are not being set high enough. With Google’s 0.0 – 1.0 scale, the expectation is to get an average of 0.6 to 0.7 across all OKRs. For organizations who are new to OKRs, this tolerance for “failure” to hit the uncomfortable goals is itself uncomfortable.”
Ben Lamorte is a coach who helps large organizations get started and sustain their OKR projects. He regularly uses a grading rather than confidence approach. From his article, A Brief History of Scoring Objectives and Key Results https://www.linkedin.com/pulse/brief-history-scoring-objectives-key-results-okrs-ben-lamorte,
“As an OKRs coach, I find most organizations that implement a scoring system either score the Key Results at the end of the quarter only or at several intervals during the quarter. However, they generally do not define scoring criteria as part of the definition of the Key Result. If you want to use a standardized scoring system, the scoring criteria for each Key Result MUST be defined as part of the creation of the Key Result. In these cases, I would argue that a Key Result is not finalized until the team agrees on the scoring criteria. The conversation about what makes a “.3” or a “.7” is also not very interesting unless we translate the “.3” and the “.7” into English.
I’ve arrived on the following guidelines that my clients are finding very useful:
Here’s an example showing the power of defining scoring criteria upfront for a Key Result.
Key Result: Launch new product ABC with 10 active users by end of Q3
Grade 0.3 = Prototype tested by 3 internal users
Grade 0.7 = Prototype tested and approved with launch date in Q4
Grade 1.0 = Product launched with 10 active users
This forces a conversation about what is aspirational versus realistic. The Engineering team may come back and say that even the 0.3 score is going to be difficult. Having these conversations before finalizing the Key Result ensures everyone’s on the same page from the start.”
As well as precision, Google sets high value on transparency. As well as all OKRs, individual and team, being posted on the intranet, team progress is shared throughout. Again, from ReWork:
“Publicly grade organizational OKRs. At Google, organizational OKRs are typically shared and graded annually and quarterly. At the start of the year, there is a company-wide meeting where the grades for the prior OKRs are shared and the new OKRs are shared both for the year and for the upcoming quarter. Then the company meets quarterly to review grades and set new OKRs. At these company meetings, the owner for each OKR (usually the leader from the relevant team) explains the grade and any adjustments for the upcoming quarter.”
And ReWork warns against the danger of set and forget:
“Check in throughout the quarter. Prior to assigning a final grade, it can be helpful to have a mid-quarter check-in for all levels of OKRs to give both individuals and teams a sense of where they are. An end of quarter check-in can be used to prepare ahead of the final grading.”
This is also done different across teams, some do a midpoint check, like a midterm grade. Others check in monthly. Google has always had an approach of, hire smart people, give them a goal and leave them alone to accomplish it. As they’ve grown OKRs are implemented unevenly, but OKRs continue to allow that philosophy to live on.
Ben Lamorte also shares a simple technique to keep OKR progress visible: progress posters. Several of his clients have set up posters in the hallway that are updated regularly with progress. Not only does this make OKRs more transparent and visible across teams, it can be effective for communicating scores on Key Results and really creating more accountability. It just doesn’t look good when your team hasn’t updated any scores and when you’re already a month into the Quarter. Most of these posters include a placeholders to update scores at 4-8 planned check-ins throughout the Quarter. Certainly OKR posters are not for all organizations, but they can be quite effective in some cases.
No matter if you use confidence check ins or formal grading (or a combination of both) there is one last piece of advice from ReWork that is important to keep in mind:
“OKRs are not synonymous with performance evaluation. This means OKRs are not a comprehensive means to evaluate an individual (or an organization). Rather, they can be used as a summary of what an individual has worked on in the last period of time and can show contributions and impact to the larger organizational OKRs.”
Use the accomplishments of each person to determine bonuses and raises. If you use the staus report system described in this report, it should be easy for each person to review their work and write up a short summary of their accomplishment. This report can guide your performance review conversations. Some things shouldn’t be automated, and the most important part of being a manager is having real conversations about what an employee has contributed. And what he hasn’t.
If you rely on OKRs results to guide you decisions, you will encourage sandbagging and punish your biggest dreamers. Reward what people do, not how go they are at working the system.
A Short Note on OKR Software
When you set a resolution, what is the first thing you do? Lose weight? Buy an expensive treadmill. Want to start running? Buy fancy shoes. Plan to diet? What is the best scale on the market? Or maybe you just buy 15 diet books. Sadly, adopting OKRs is treated the same way. People buy software, and hope it’ll do the hard work of setting and managing your goals.
There are a ton of tools for OKRs out there, and many are quite good. But buying a tool is the last step you want to take, not the first. The right way to adopt OKRs is to adopt them in a lightweight fashion, then experiment with different approaches until you find the system that gets you results.
Start with these tools first:
• A whiteboard, to write ideas of what your objectives will be.
• Post-it notes, to brainstorm good KRs.
• Powerpoint, to track confidence and efforts against the objectives.
• Email, to send statuses out.
Excel, if you decide you wish to do formal grading. (Google offers a tool for grading on their Rework website. https://rework.withgoogle.com/)
The first quarter you feel you have truly mastered OKRs, then go shopping.