Tierney, W. (Warren)

Filtering

2020 - 20212

Settings

Author search.filters.isAuthorOfPublication.14df2710-591b-4212-96c6-866a994b1e69

Search Results

Now showing 1 - 2 of 2
  • Thumbnail Image
    Creative destruction in science
    (Elsevier, 2020-09-29) Hardy-III, J.H. (Jay H.); Ebersole, C. R. (Charles R.); Viganola, D. (Domenico); Pfeiffer, T. (Thomas); Dreber, A. (Anna); Hiring Decisions Forecasting Collaboration; Leavitt, K. (Keith); Uhlmann, E. L. (Eric Luis); Tierney, W. (Warren); Clemente, E. G. (Elena Giulia); Gordon, M. (Michael); Johannesson, M. (Magnus)
    Drawing on the concept of a gale of creative destruction in a capitalistic economy, we argue that initiatives to assess the robustness of findings in the organizational literature should aim to simultaneously test competing ideas operating in the same theoretical space. In other words, replication efforts should seek not just to support or question the original findings, but also to replace them with revised, stronger theories with greater explanatory power. Achieving this will typically require adding new measures, conditions, and subject populations to research designs, in order to carry out conceptual tests of multiple theories in addition to directly replicating the original findings. To illustrate the value of the creative destruction approach for theory pruning in organizational scholarship, we describe recent replication initiatives re-examining culture and work morality, working parents’ reasoning about day care options, and gender discrimination in hiring decisions.
  • Thumbnail Image
    Same data, different conclusions: Radical dispersion in empirical results when independent analysts operationalize and test the same hypothesis
    (Elsevier, 2021) Strobl, C. (Carolin); Canela, M.A. (Miguel A.); Viganola, D. (Domenico); Staub, N. (Nicola); Schaumans, C.B.C. (Catherine B.C.); Murase, T. (Toshio); Snellman, K. (Kaisa); Goldstein, P. (Pavel); Althoff, T. (Tim); Schweinsberg, M. (Martin); Kelchtermans, S. (Stijn); Bernstein, A. (Abraham); Feldman, M. (Michael); Aert, R.C.M. (Robbie C.M.) van; Tierney, W. (Warren); Robinson, E. (Emily); Heer, J. (Jeffrey); Sommer, S.A. (S. Amy); Madan, N. (Nikhil); Prasad, V.V. (Vaishali Venkatesh); Mandl, B. (Benjamin); Liu, Y. (Yang); Kale, A. (Alex); Amireh, H. (Hashem); Robinson, D. (David); Silberzahn, R. (Raphael); Akker, O.R. (Olmo R.) van den; Assen, M.A.L.M (Marcel A.L.M.) van; Otner, S.M.G. (Sarah M.G.); Mohamed, Z. (Zainab)
    In this crowdsourced initiative, independent analysts used the same dataset to test two hypotheses regarding the effects of scientists’ gender and professional status on verbosity during group meetings. Not only the analytic approach but also the operationalizations of key variables were left unconstrained and up to individual analysts. For instance, analysts could choose to operationalize status as job title, institutional ranking, citation counts, or some combination. To maximize transparency regarding the process by which analytic choices are made, the analysts used a platform we developed called DataExplained to justify both preferred and rejected analytic paths in real time. Analyses lacking sufficient detail, reproducible code, or with statistical errors were excluded, resulting in 29 analyses in the final sample. Researchers reported radically different analyses and dispersed empirical outcomes, in a number of cases obtaining significant effects in opposite directions for the same research question. A Boba multiverse analysis demonstrates that decisions about how to operationalize variables explain variability in outcomes above and beyond statistical choices (e.g., covariates). Subjective researcher decisions play a critical role in driving the reported empirical results, underscoring the need for open data, systematic robustness checks, and transparency regarding both analytic paths taken and not taken. Implications for organizations and leaders, whose decision making relies in part on scientific findings, consulting reports, and internal analyses by data scientists, are discussed.