Max Blumberg
A number of people suggest there is no difference between academic and organistional research and that conclusions from academia can be easily transported to organisations. I argue that this is untrue for the following reasons:
* In the academic world, we optimise the experimental environment to achieve MAXMINCON: MAXimise variance due to the independent variable(s), MINimise error variance, CONtrol nuisance variables and extraneous variance. But the real world is not some controllable experiment where one can reconfigure the client’s organisation to achieve MAXMINCON. Circumstances change half-way through an exercise (economy, new CEO, etc. – all these impact measurements and interventions).
* The sample sizes available in the real world are often too small to deliver the required power to draw the heady conclusions available in academic research (where we keep on building the sample size until it is large enough)
* Error variances are seldom normally distributed in the real world and are related. Most techniques learned in academic programmes are based on a General Linear Model which when applied in the real world lead to inflated alpha and beta errors – or in English, are often not valid. Different analytical techniques are required in the real world in order to deliver ‘evidence’.
* Conclusions gleaned from academic research are valid under the same controlled environment; making claims that they can be generalised to real world organisational environments is irresponsible. For example, if one finds in academic research that a given competency framework results in higher performance in an experimental group relative to a control group, one cannot simply sell this framework to every organisation claiming it will work as well. Workforces, cultures, regions and buiness units differ.
* In the academic world, even small effect sizes are acceptable as long as the result is significant (say p < .05). Say you find a correlation of r = .20 (p <.01) between engagement and performance. In the academic world, this is a publishable win even though only 4% of the variance in performance is explained by engagement. In the real world, are you really going to suggest to head of HR they should invest in an expensive engagement programme – knowing that engagement accounts for only 4% of performance? I often see this being done in the real world and it is at worst unethical and at best shows non-understanding of the difference between significance and effect size.
Bottom line: One needs to be extremely circumspect about claiming to deliver evidence-based I/O services in the real world.
View Rainmakers Bio »
Go to Source »
