Training Value Metrics: 4 Progressive Levels

Pro Tip #1: Don't wait for a perfect measurement system before you start training. If you have a reasonable amount of experience and situational context, start with the most basic training you can start now. Study the results and make continuous improvements to the training and to the standards. 

Consider training value metrics in four progressive levels. 

1. Was the training attended? (Yes or No - the most basic and tangible metrics)
2. Did the attendees enjoy and perceive value from the training? (Self-evaluations and observations - subjective)
3. Was there a noticeable difference in knowledge, beliefs, behaviors, or capabilities after the training? (1 week and 3 months after - as observed and/or measured by others)
4. Was there a measurable difference in the business outcomes you were looking to improve?

These four progressive levels move from extremely tangible and immediate feedback to longer-term feedback with lots of additional variables.

The fastest way to waste a lot of money on training is to focus only on measuring what is easy. This nearly always happens with a contractor's first pass at building an internal training function that is separate from operations, usually tucked under HR. This is the right direction so if you are struggling with it, please don't overreact and set progress back even further. Training is especially crucial in the construction industry up through 2030 as the workforce is rebuilt. 

"Not everything that can be counted counts, and not everything that counts can be counted." - William Bruce Cameron

Begin With the End in Mind - Habit 2 (The 7 Habits of Highly Effective People)

While levels 1 and 2 are the easiest and fastest to measure, it is the impact on outcomes that you are looking for. That starts with knowing what you really want to change. Start by looking at your market strategy and capabilities, your business model, your organizational structure and strength, your 3-year business plan, and your scoreboards and scorecards starting at the highest level. 

Challenge: Define your top 3-5 most impactful trainings over the next 12 months using a sentence structure similar to the example below. One sentence per training. 

[Training Description] for [Audience Description] is expected to move [Scoreboard Metric Name] from [Current Measurement] to [Future Target Measurement] by [Date].

Example: Cash flow training for all project managers is expected to move our cash metric from 0.87 to 1.1 by the end of the year. 

Pro Tip #2: There are measurably fewer people who can turn concepts into concrete and consistent actions. For rapid and broad adoption, focus the majority of your training on developing skills and competency levels. Let your management systems integrate those competencies into organizational capabilities and outcomes. 

"It's easier to act your way into a new way of thinking, than think your way into a new way of acting." Jerry Sternin, The Power of Positive Deviance.

• Identify the all the specific activities (tasks & decisions) that are connected to the outcome (cash flow example).
• Looking at your current state, which of those activities, if improved, will have the biggest positive impact on the outcomes?
• Prioritize based on an optimum combination of need to, can do, and will do. 
• Design your training around those priorities keeping the focus narrow. You will get more results from focusing on leveling up a whole team's competency level on 1-3 activities at a time than to try to improve performance on a dozen activities. 

Example: Continuing with the cash flow example, the training may now focus on a few specific activities:

1. Change identification, pricing, and approval.
2. Building a Schedule of Values (SOV) and getting it approved
3. Progress billing targets and customer approvals. 

Pro Tip #3: Your standards, tools, projects, and metrics must be the basis for most training.

"Without standards there can be no improvement." - Taiichi Ohno, "Father" of the Toyota Production System

All standards start with an experienced person giving direction and a thumbs up or down. They progress from there down to explicitly defined installation standards for the highest-volume and highest-value assemblies. 

• Set the standard - even if it is only a hand sketch and a few bullet points. 
• Train to the standard including tips, tricks, and other nuances. 
• Perform to the standard. 
• Study the performance and results.
• Provide feedback to improve the person, the training, and/or the standard as required.

This basic method dramatically improved productivity and capacity during World War II while facing huge labor shortages. Many aspects of that system were adopted by Toyota to become the largest car manufacture in the world. 

Example: Looking at the change identification, pricing, and approval process. 

1. Checklist of the most common changes on projects sorted by a combination of highest frequency and impact. Set time during project planning where that checklist is used to review the contract, scope, plans, and specification. Train on the checklist - what each item is and where to look for them. 
2. Standard change pricing tools and guidelines including labor rates, material pricing, quotes, mark-ups, scope, and maximum timing from identification through submission to the customer. An initial level of written standard for this can be developed in hours and is far better than "Tribal Knowledge." Train on pricing up some real-world change examples to ensure everyone can use the tools as expected and develop with consistent change proposals. 
3. Change management tracking and summary reporting with a standard operating rhythm for when different levels of job roles are expected to review and ensure all changes are moving forward. This standard would include triggers for escalations based on factors like timing, total change size, first-of-a-kind changes, and costs-at-risk. Training would focus on how to use and read the different reporting along with what, why, and how for the escalations including some relevant examples from the company. 

Pro Tip #4: Look for quick wins and build on that momentum to build foundational competencies. The more complex capabilities and understanding nearly always emerge from there. 

Building competency levels through training requires deliberate practice. Deliberate practice is very difficult and only becomes enjoyable for most people when that hard work is connected to winning. 

"The greatest threat to success is not failure but boredom. We get bored with habits because they stop delighting us... you have to fall in love with boredom." - James Clear, Atomic Habits

There is a reason for the hundreds of movies, books, and stories similar to The Karate Kid. They are built on human nature. This must be taken into account whether you are the trainer or trainee. The arc is always familiar. 

• A problem or opportunity presents itself (to the trainee)
• The trainee can visualize what they want or don't want for results. 
• The trainee doesn't know what they don't know - the unconscious incompetence stage of learning a new skill.
• The trainee will try on their own without training. For some tasks, this can be expensive and dangerous, even fatal.  
• The failure prompts trainee to seek training.
• An experienced trainer starts with the basics including lots of repetition and feedback. 
• The trainee gets bored and frustrated, eventually stopping the training and running away from the trainer. 
• The trainee gets attracted to a different trainer promising shortcuts. 
• The trainee thinks they are ready (over-confidence) and makes another attempt on their own with similar results.
• The trainee goes back to the original training program and trainer doubling down on their practice.
• The trainee starts putting all their individual competencies together into capabilities - they are ready and prove this with results. 
• The trainee begins to innovate on that basic foundation taking themselves to a new level of performance.
• The trainee becomes the trainer building others.

Whether it is Daniel learning from Mr. Miyagi or Luke Skywalker learning from Yoda or an apprentice learning from an experienced journeyman, the cycle is nearly always the same. As you are designing the training program, if you don't take this into account, you will never achieve your desired outcomes. 

Geoff Colvin, author of Talent is Overrated describes an example of effective training including both the dislike of the training and results - nearly a 4X increase in performance.  

Example: Use a series of five completed projects with progressively more complex scopes along with a checklist for RFIs and Changes. Run people through the review and feedback process until everyone on the project team from estimators to project managers and foremen is identifying 95% or more of what can be identified up front.

• Is this costly and time consuming? Absolutely. 
• Will they be irritated? Very likely.
• Is this a lot cheaper than identifying RFIs and Changes late, including the impact on customer satisfaction? Probably 10X Cheaper.

• How much does slow change processing cost you in cash flow?
• How many times does late identification of changes put you in a bad negotiating position - including not getting the changes approved or having to settle significantly on the costs?
• How many times have you seen changes after the project is completed and can't recover your costs?
• How much do unanswered RFIs impact your field productivity?

Conflicting Metrics: This is the interesting dichotomy of training value metrics. This is where there is often an inverse relationship between how much someone enjoyed as compared to their ability to do the work consistently at or above the standard months after the training was completed.

The training can't be so challenging that no one goes - but be prepared that attendance, enjoyment, and engagement metrics are not the same as outcome metrics. You have to dig into the why behind them to understand if you have to make adjustments to the training, or if that little bit of discomfort and rebellion is just part of the process. 

In most situations, the trainee won't fully understand the rationale until they have moved into a management position for a couple years and are successful at the two basic accountabilities -  

1. Delivering consistent outcomes given the inconsistencies of the inputs, people, and environment.
2. Developing people at all levels into their fullest potential along the way.

Pro Tip #5: The best training is usually frequent and actionable feedback from the manager. 

"A coach is someone who can give correction without causing resentment." - John Wooden

Most failed training is results from lack of clarity about the role of a manager including what standard work they are expected to do for Quality Control (QC - right outcomes) and Quality Assurance (QA - right process). 

The work observations required for QA/QC will result in continually better standards and rapid development of people. 

Example: 

1. There is a standard tool for Schedule of Values (SOV) that cross-references the project budget and schedule milestones. When the billing and payment terms are added from the contract, the tool provides a quick estimate of the project cash flow. 
2. The Project Manager is directly responsible for completing this internally within 8 days of project award and submitted for internal approval along with supporting documents, assumptions, and notes. 
3. The manager must review, provide feedback, and adjust as necessary to meet company cash flow targets within 2 days. Direct responsibility for review, approval, and exception limits:
   1. Projects under $1M: Project Executive
   2. Projects $1-5M: Operations Manager
   3. Projects over $5M: COO
   4. Notify the CFO about any projects where the projected cash flow will not meet company standard targets

This is just an example and not meant to be prescriptive for every contractor. Setting up the initial standards and tools can be done in a few hours and improved over time. 

• If this is consistently executed, it will have a dramatically bigger impact on cash flow and project outcomes than any amount of training. 
• If there was training based on the SOV standard and tool without reinforcement by the managers of the PMs, the outcomes will be negligeable - especially six months after the training. 

See a similar example for Short-Interval-Planning (SIP). 

That covers a lot - remember Pro Tip #1 and don't let the need for perfect get in the way of progress.

1. Look at current outcomes that you are responsible for and pick one. 
2. Observe one or two of your team working on something related to that outcome.
3. Give some actionable feedback. 
4. Spend 30 minutes creating or improving a standard or tool.
5. Repeat.

Like a diet or exercise - the best one is the one you will stick with consistently.