The question, stated fairly
Every budget cycle, some version of this question lands on a CHRO’s desk: we spend real money developing managers — what do we get for it?
It is a fair question, and it deserves a straight answer rather than a defensive one. Leadership development is an unusual line item. The output is a behavior, not an asset. The payback arrives late, if it arrives at all. And the two parties best positioned to evaluate the program — the vendor who sold it and the HR team who bought it — both have an interest in concluding that it worked.
Most of the evidence offered in response makes the problem worse. Vendor case studies, single-client testimonials, and research published by the firm selling the service are not evidence a finance partner should accept, and a CHRO who brings them into a budget meeting should expect to lose the argument. The honest answer starts by conceding that much of what circulates as proof is marketing.
There is better evidence. It has been in the peer-reviewed literature since 2017, and it does something more useful than settle the yes-or-no question.
What 335 studies show
Lacerenza, Reyes, Marlow, Joseph, and Salas published a meta-analysis in the Journal of Applied Psychology that pooled 335 independent samples covering 26,573 participants (Lacerenza et al., 2017). The source studies reach back to 1951, include both published and unpublished work, and were restricted to programs delivered to working employees rather than MBA or undergraduate students. That last screen matters: it means the sample describes practicing managers, not simulations of them.
The authors measured effects against four criteria — reactions, learning, transfer to the job, and organizational results. Corrected for unreliability in the criterion, leadership training improved reactions (δ = .63), learning (δ = .73), transfer (δ = .82), and results (δ = .72). The overall effect was δ = .76, 95% CI [.64, .89] (Lacerenza et al., 2017).
Two features of that finding matter more than the headline. First, the authors tested for publication bias three ways, including a trim-and-fill analysis that imputed zero missing studies, and found no evidence of it; published and unpublished studies did not differ significantly. Second — and this is the part vendors leave out — the four criterion effects were not statistically different from one another. The paper’s own conclusion is that leadership training programs tend to be equally effective across criteria. Do not let anyone sell you the .82 transfer figure as evidence that behavior change is where the returns concentrate. The data do not support that ranking.
Translating the number without inflating it
An effect size is not a percentage and not a dollar figure. Delta expresses a difference measured in standard deviations. Under Cohen’s widely used convention, .20 is small, .50 is medium, and .80 is large, which places these results in the medium-to-large band (Cohen, 1988).
A more concrete reading, using the standard percentile conversion and assuming roughly normal outcome distributions: an effect of .73 puts the average trained manager at about the 77th percentile of the untrained comparison group. A manager who sat in the middle of the pack before training performs better than roughly three-quarters of their untrained peers afterward. That is a real and visible change. It is not a miracle, and it is not an ROI multiple — the study measures effect, not payback period, and anyone converting these deltas directly into a dollar return is doing arithmetic the paper does not license.
One honest caveat belongs in the same breath. The paper reports a moderator for study design, and the most rigorous designs — those combining independent groups with repeated measures — produced a smaller effect, δ = .48, 95% CI [.25, .71] (Lacerenza et al., 2017). The effect stays clearly positive under the tightest methodology. It also shrinks. A CFO who asks whether the headline is flattered by weaker study designs is asking a good question, and the answer is: partly yes, and the finding survives anyway.
The design conditions that separate the programs that work
The reason to bring this paper into a budget conversation is not the average. An average across 335 programs describes a market, not a purchase. What makes the study useful is that it tests which design choices move the effect — which turns a defense of the category into a specification for buying inside it.
The moderator analyses support six conditions (Lacerenza et al., 2017):
-
Needs analysis. Programs built on a documented analysis of the organization’s actual gaps outperform generic curricula.
-
Feedback. Programs that give participants structured feedback outperform programs that do not.
-
Multiple delivery methods, especially practice. Information alone underperforms information combined with demonstration and practice.
-
Spaced sessions. Content distributed across multiple sessions outperforms the same content massed into a single event.
-
On-site location. Training delivered in or near the work environment outperforms off-site delivery.
-
Face-to-face delivery that is not self-administered. Programs facilitated by an instructor outperform self-administered ones.
A note on precision, because this paper is widely misquoted. The direction of each of these six findings is well supported. The specific contrast figures that circulate on vendor websites — “d = 0.78 with needs assessment versus 0.41 without,” and similar pairings — do not correspond to the values in the published tables, and we will not repeat them. Several of the paper’s own moderator estimates carry very wide confidence intervals, which is exactly why selective quotation of them is so easy and so misleading. Use the direction of the findings. Do not quote the contrast deltas.
Read that list again as a procurement checklist
Now hold the market against those six conditions.
The two-day leadership retreat at a conference center fails spacing, fails on-site location, and usually fails needs analysis. The self-paced e-learning library fails facilitation, fails feedback, and fails practice. The keynote speaker fails all six. The off-the-shelf curriculum delivered identically to every client fails needs analysis by design — that is what “off-the-shelf” means.
This is the finding that should change the budget conversation. The question is not whether manager development works. It does, across every criterion the research measures. The question is whether the specific program in front of you was built the way the effective ones were built. Most programs on the market fail three or more of the six conditions, which means the typical buyer is purchasing something meaningfully weaker than the average the literature reports — and then citing that average to justify it.
That reframing gives a CFO something more useful than a verdict on the category. It gives a basis for cutting the spend that will not work and defending the spend that will. It also gives an L&D function a defensible standard to hold vendors to, which is a better position than arguing about whether development matters in the abstract.
What to ask a vendor — including us
Six questions follow directly from the evidence. Ask them of every provider on your list, and ask them of gothamCulture:
-
What needs analysis will you conduct before designing this, and what document will it produce?
-
What feedback will participants receive, from whom, and how many times?
-
What proportion of contact time is practice rather than presentation?
-
Across how many weeks is this delivered, and in how many separate sessions?
-
Where does it happen, and how close is that to where the work actually happens?
-
Who facilitates, and what is the ratio of facilitated time to self-administered time?
A provider who cannot answer these is not necessarily selling a bad program. They are selling a program whose design they cannot defend against the strongest evidence in their own field, which is a different problem and not a smaller one.
Ask us the same six questions. Where the honest answer is weak for what your organization actually needs, we would rather say so before you buy than explain it afterward.
Sources
Lacerenza, C. N., Reyes, D. L., Marlow, S. L., Joseph, D. L., & Salas, E. (2017). Leadership training design, delivery, and implementation: A meta-analysis. Journal of Applied Psychology, 102(12), 1686–1718. https://doi.org/10.1037/apl0000241
Full text (PDF, Rice University): https://doerr.rice.edu/sites/g/files/bxs5941/files/2026-01/Leadership%20Training%20Design,%20Delivery,%20and%20Implementation.pdf
PubMed record (PMID 28749153): https://pubmed.ncbi.nlm.nih.gov/28749153/
Rice University MET Lab summary of the findings: https://metlab.rice.edu/2017/12/06/meta-analysis-identifies-the-conditions-under-which-leadership-training-is-most-effective/
Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Lawrence Erlbaum Associates. (Source of the .20 / .50 / .80 small–medium–large convention referenced above.)