The February 2010 edition of T+D magazine printed an article on evaluating training (read the article here). The article highlighted the fact that only 36.9% evaluated the results of training, but 75.0% said evaluation has a very high value.

The application of Lean Six Sigma Methods can ensure that you evaluate the results.

The barriers listed are; 1) Too difficult to isolate training’s impact on results versus other factors influence, 2) our LMS does not have a useful evaluation tool, 3) Evaluation data is not standardized enough to compare well across functions, 4) it costs too much to complete higher levels of evaluation, 5) leaders don’t generally care about evaluation data, 6) the evaluations are too difficult to understand for most people, and 7) the evaluations are not seen as credible.

Training is a process. There exists a logical flow of needs analysis, task analysis, course design, short term evaluation and long term evaluation.

Much training is “Front Loaded”. We determine the tasks, train on those, and evaluate. It would be far better to focus on the results of the training. “What does this organization want to accomplish with this training?” If you can get the needs of the organization aligned to results, everyone will fully understand the purpose.

Lean Six Sigma methods begin with project Definition Stage.  Executive sponsors of the project determine what is critical to cost, quality or schedule. A project charter defines the objective, and the key business need is addressed. This addresses one of the seven (#5 above) barriers, “Leaders generally don’t care about evaluation data.”  By defining the project scope, they are indicating what they do care about.

In the Measurement Stage, “Dashboard Metrics” are designed to be reviewed at a glance.  It is possible to develop common language and standardize that language across different functions.  This addresses the second and third barriers (#3 & 6 above), “Evaluation is too difficult to interpret for most people”, and, “Evaluation data is not standardized enough to compare well across functions”.

Also in the Measurement Stage, the process is defined.  This is not limited to the training department. Here you document formal and informal processes to identity feedback on the job, loss of retention, or job aids.

During the Analyze Stage, the team identifies sources of variation. Statistical tools determine the impact of training by itself, or with other factors.  The fourth of seven barriers (#1 above) has been addressed, “Too difficult to isolate training’s impact on results versus other factors influence.”

Also in the Analyze Stage, the team uses statistically methods that relate directly to the results. Lean Six Sigma is data driven.  The fifth and sixth barriers (#2 & 7 above) , “our LMS does not have a useful evaluation tool” and “evaluations are not seen as credible”, are addressed.

The Improve Stage establishes the required changes to the process and documents the effectiveness of the change.

In the Control Stage, the team makes the changes permanent and completes the final report, including the financial return on investment.

Lean Six Sigma methodology typically results in savings by reducing the cost of quality.  Cost of quality is buried in operating costs. There are costs tocorrect failures and costs to prevent errors.  It costs a relatively minor amount to prevent an error, much more when it is detected in-house, and significantly more if detected by the customer.

A typical organization can move from 3 sigma to 4 sigma and reduce 91% of errors, generating significant cost savings.

This article has established precisely where Lean Six Sigma methodologies address the seven “Barriers to the evaluation of learning organizations”. The concern that “Leaders generally don’t care about evaluation data” is addressed in the Definition Stage by requiring them to set the direction of desired results. The next two concerns, “Evaluation data is too difficult to interpret for most people”, and “Evaluation data is not standardized enough to compare well across functions” are addressed in the Measurement Stage.  Concerns stated as, “Too difficult to isolate training’s impact versus other factors influence” is addressed using tools from the Measurement Stage and the Analyze Stage.  The statistical tools in the both the Analyze Stage and the Improve Stage eliminate the concerns, “our LMS does not have a useful evaluation tool” and “evaluations are not seen as credible”. Finally, through proper project definition, reduction in the cost of quality, and proper project closure in the Control Stage, we can generate the funds to address, “it costs too much to conduct higher level evaluations”.

Would you like to make comments about this article? Visit our LinkedIn group for discussions about this topic and more by clicking here.

Are you interested in publishing an article about a recent book you've read, event you've attended, or  subject you're passionate about in an ASTD MT Newsletter? Keep it to less than 750 words (ideally less than 500) and send it to info@astdtn.org and it may get published for over 800 learning professionals in Middle Tennessee.