I will now continue where I left off in the previous Element K posting, "Increasing Discussion of Learning Evaluation and ROI, Part 1." In this part two of two, I will describe and link to the two most provocative articles I've seen on this general topic in the past few months.
In the June 2009 issue of Training magazine, industry book author Saul Carliner gets a bit more provocative with his article "Maybe ROI Really Is a Waste of Time." He notes that one hurdle with ROI for training is that often "senior management is not interested in the evaluation because training isn't a significant part of their portfolio." That is, the total cost of training might be so small, relatively speaking, that cutting its budget will not have a significant cost-reduction impact on the organization. But Carliner then notes "That doesn't mean organizations lack interest in seeing a return on their investment in training. But the effort of showing how each course returned ROI provides little ROI—and that's why senior organizational managers don't actively seek it as much as some of the trade press might have us believe. "
He then raises the importance of non-quantitative approaches to training evaluation: "As suggested by the C-level study mentioned earlier, perceptions play a significant role in assessing the training function. Indeed, the survey by my research team found that word of mouth was the most common measure of effectiveness that training managers felt was important to their sponsor, ranking higher than ROI and course evaluations. "
Carliner concludes his "soapbox" column in this issue with the following advice: "In short, perhaps trainers should stop trying to collect ROI on individual courses because information about the ROI of individual courses requires resources that are rarely available to collect; does not provide a credible measure; and, most significantly, senior managers are not asking for it. Furthermore, because perceptions of training seem to have more credibility—and are more challenged among senior managers—and because other financial data is both more credible and provides more established evidence of the actual return on training, perhaps trainers instead should focus efforts on tracking and managing perceptions and following the broader measures that provide insights into the effectiveness of an organization's total investment in training rather than investments in individual courses."
And finally, in the July 2009 issue of Chief Learning Officer magazine, Jay Cross and Jon Husband take the discussion of ROI in a truly path-breaking direction with their article "Productivity in a Networked Era: Not Your Father's ROI." The authors begin by setting up the discussion: "The network era now replacing the industrial age holds great promise. Networked organizations are reaping rewards for connecting people, know-how, and ideas at an ever-faster pace. Value creation has migrated from what we can see (physical assets) to intangibles (ideas). …Understandably, seasoned executives, chief learning officers among them, are having a devil of a time shifting from the industrial age mindset of logic, certainty, and bounded constraints to the network gestalt of interaction, self-organization, unpredictability, and fewer limits to potential. …We are shifting into an era in which knowledge work and learning occur where re-engineered business processes collide with a participative and interactive ecology of information flows."
Cross and Husband then introduce a new concept as follows: "In an environment of continuous flow and interaction, there’s a need to consider an emerging metric: return on investment in interaction (ROII). The working definition of ROII is the observable development of capacity and capability to create economic values out of intangibles."
In the rest of the article, the authors describe what traditional ROI was, persuasively convince the reader of the increasing value of intangibles, and note the inherent clash between this trend and the demand for ROI metrics. They describe the essential characteristics of business networks, and argue that ROI is simply not up to the task. They then give some assumptions for their analysis of ROII and describe some of the potential components that would be quantifiable. In doing so they include illuminating examples from such companies as Cisco, Ford, and Capgemini. In closing, they frame the challenge as being: "Network returns are asymmetric, so simplistic count-’em-up approaches are no longer viable. But how can one make a solid network-era case to an executive who is still playing by yesterday’s rules?" Read this entire article to learn their initial advice on how to answer that question, and to understand this ground-breaking conceptual proposal for the L&D industry.
— Thomas Stone (Tom_Stone@elementk.com)
Comments