Our Faculty and Staff

< Back to list

Benjamin M Kelcey

Title: Associate Professor of Quantitative Research Methodologies
Office: 3311B Teachers College
Tel: 513-556-3608
Email: kelceybn@ucmail.uc.edu

My research focuses on causal inference and measurement methods within the context of multilevel and multidimensional settings such as classrooms and schools. With respect to causal inference, I develop and apply methods for statistical adjustment that enable researchers to mount focused, specific analogies of their observational studies to randomized experiments with emphasis on multilevel or clustered settings. In this way, I view statistical methods not as substitutes for random assignment but rather as tools to uncover objective information that will lead to well specified and designed randomized studies. My measurement work is also generally situated within multilevel contexts (e.g., measurement of teaching) and develops and applies methods to address the multilevel and multidimensional nature of teaching and learning. Statistically, these research areas unfold within the context of multidimensional, multilevel and cross-classified explanatory item response models, mixed effect models, propensity score methods, and analogues and extensions of each of these. The confluence of these statistical foci take substantive root in understanding the extent to which teacher and school factors can intervene to promote more effective and contextually responsive classroom and school environments. This work has included the measurement and analysis of teachers' knowledge, instructional quality and strategies, and student achievement to identify and develop methods of effective teaching and to establish links among these constructs.

Education

  • Ph.D., University of Michigan, 2009 (Quantitative Research Methods).
  • M.A., University of Michigan, 2006 (Statistics).
  • B.S., University at Albany, 2001 (Mathematics).

Research Information

Research Support

  • Multisite Designs for Developmental Processes in Mathematics, National Science Foundation. 2018 to 2021.
  • Multilevel Mediation Models to Study the Impact of Teacher Development on Student Achievement in Mathematics, National Science Foundation. 2016 to 2021.
  • Project Coordinate: Increasing Coordination and Use of Evidence-based Practices (with M. Brownell), Institute of Education Sciences. 2017 to 2021.
  • Exploration of Writing Instruction for Kindergarten Children (with Ying Guo, Allison Breit-Smith, and Chris Swoboda), Institute of Education Sciences. 2016 to 2020.
  • Optimizing Learning Opportunities for Students' Early Learning Observation System (with Carol Connor), Institute of Education Sciences. 2016 to 2020.
  • The Day Reconstruction Method: A New Tool for Measuring the Work of K-12 Teachers, and the Contexts of Their Work (with E. Camburn), Institute of Education Sciences. 2016 to 2020.
  • Power Analyses for Moderator and Mediator Effects in Cluster Randomized Trials (with Jessaca Spybrook & Nianbo Dong), National Science Foundation. 2014 to 2018.
  • ..
  • Measuring Teaching with Cross-Classified Random Item Effects Item Response Models, William T. Grant Foundation. 08-15-2013 to 07-31-2014.
  • Exploring Effective Reading Comprehension Instruction: Classroom Practice, Teacher, and Student Characteristics (with Carol Connor and Joanne Carlisle), Institute of Education Sciences. 2013 to 2016.
  • A data-driven approach to kindergarten readiness and the importance of the preschool years: A partnership between researchers, practitioners, and stakeholders (with Jessica Logan), Ohio Education Research Center. 2013 to 2014.
  • Measuring Teaching with Cross-classified Random Item Effects Item Response Models, William T. Grant Foundation and the Spencer Foundation. 2013 to 2014.
  • Empirical Benchmarks of Design Parameters for Group Randomized Trials (with Geoffrey Phelps & Nathan Jones), National Science Foundation. 2012 to 2016.
  • The Sensitivity of Classroom Observation Assessments of Teacher Quality to Scoring Model (with Heather Hill), National Center for Teacher Effectiveness. 2012 to 2013.
  • Empirical Benchmarks of Design Parameters for Group Randomized Trials , National Science Foundation. 2012 to 2016. Status: Active.
  • Multilevel Measurement Models for Understanding Teacher Quality (with Brian Rowan). 2011 to 2012.
  • Development of an Interactive, Multimedia Assessment of Teachers' Knowledge of Early Reading (with Joanne Carlisle), Institute of Education Sciences. 2010 to 2013.
  • Investigating the Measurement and Development of Mathematical Knowledge for Teaching (with Geoffrey Phelps), National Science Foundation. 2009 to 2012.
  • Assessment of Pedagogical Content Knowledge of Teachers of Reading (with Joanne Carlisle), Institute of Education Sciences. 2009 to 2011.
  • Mission Possible: Graudation (MPG) King-Chavez Parks (KCP) Initiative (with S. Kanoyton), State of Michigan Dept of Energy. 2009 to 2010.
  • Measuring Teacher Instruction (with S. Kanoyton), State of Michigan Dept of Energy. 2009 to 2010.
  • ..., . ....
  • ..., .
  • ., ..

Publications

Peer-reviewed Publications

  • Kelcey, B. (2018). A Robust Alternative Estimator for Small to Moderate Sample Structural Equation Models https://doi.org/10.1016/j.addbeh.2018.10.032. Addictive Behaviors.
  • Kelcey, B., Spybrook, J., & Dong, N. (2018). Sample Size Planning in Cluster-randomized Studies of Multilevel Mediation. Prevention Science.
  • Cox, K. & Kelcey, B. (2018). Optimal Sample Allocation in Group-randomized Studies of Multilevel Mediation with a Group-level Mediator. Journal of Experimental Education.
  • Kelcey, B. & Shen, Z. (2018). Strategies for Efficient Experimental Design in Studies Probing 2-1-1 Mediation. Journal of Experimental Education.
  • Kelcey,B. Dong, N., Spybrook, J. & Cox, K. (2017). Statistical Power for Causally-defined Indirect Effects in Group-randomized Trials with Individual-level Mediators . Journal of Educational and Behavioral Statistics.
  • Kelcey, B. & Shen, Z. (2017). Planning community-based assessments of HIV educational intervention programs in sub-Saharan Africa. Health Education Research.
  • Kelcey, B., Dong, N., Spybrook, J. & Shen, Z. (2017). Experimental Power for Indirect Effects in Group-randomized Studies with Group-level Mediators . Multivariate Behavioral Research.
  • Kelcey, B., Phelps, G., Spybrook, J., Jones, N., & Zhang, J. (2017). Designing Large-scale Multisite and Cluster-randomized Studies of Professional Development. Journal of Experimental Education.
  • Kelcey, B. & Shen, Z. (2017). Experimental evaluations of school-based HIV programs in sub-Saharan Africa. Studies in Educational Evaluation.
  • Shen, Z., Escue-Simon, C., & Kelcey, B. (2017). The potential consequences of using value-added models to evaluate teachers. eJournal of Education Policy.
  • Dong, N., Kelcey, B., & Spybrook, J. (2017). Power analyses for moderator effects in three-level cluster randomized trials. Journal of Experimental Education.
  • Zaidi, Nikki L Bibler; Swoboda, Christopher M; Kelcey, Ben; Manuel, R Stephen (2017). Hidden item variance in multiple mini-interview scores.. Advances in health sciences education : theory and practice, 22 (2), 337-363.
  • Spybrook, J. & Kelcey, B. (2016). Introduction to Three Special Issues on Design Parameters Values for Planning Cluster Randomized Trials in the Social Sciences. Evaluation Review, 40, 491-499.
  • Wallace, T., Kelcey, B., & Ruzek, E. (2016). What Can Student Perception Surveys Tell Us About Teaching?: Empirically Testing Theoretically-Proposed Dimensions of Teaching. American Educational Research Journal.
  • Spybrook, J., Kelcey, B., & Dong, N. (2016). Power Analyses for Detecting Treatment by Moderator Effects in Cluster Randomized Trials. Journal of Educational and Behavioral Statistics.
  • Phelps, G., Kelcey, B., Liu, S., & Jones, N. (2016). Performance Trajectories of Teacher Knowledge Growth Throughout Professional Development Programs.. Evaluation Review.
  • Kelcey, B, Shen, Z., Spybrook, J. (2016). Intraclass Correlation Coefficients for Designing School Randomized Trials in Education in Sub-Saharan Africa. Evaluation Review.
  • Kelcey, B. & Shen, Z. (2016). Multilevel Design of School Effectiveness Studies in Sub-Saharan Africa. School Effectiveness and School Improvement.
  • Spybrook, J., & Shi, R., & Kelcey, B. (2016). Progress in the Past Decade: An Examination of the Precision of Cluster Randomized Trials Funded by the U.S. Institute of Education Sciences. International Journal of Research & Method in Education.
  • Dwyer, J. Kelcey, B., Berebitsky, D., & Carlisle, J (2016). A Study of Teachers’ Discourse Moves That Support Text-based Discussions. Elementary School Journal..
  • Adelson, J. & Kelcey, B. (2016). Bringing a perspective from outside the field: A commentary on Davis et al.’s (2010) use of a modified regression discontinuity design to evaluate a gifted program.. Journal of Advanced Academics.
  • Kelcey,B. (2014). Measuring Teachers’ Instruction with Multilevel Item Response Theory.. International Journal of Research in Education Methodology.
  • Moss, B., Kelcey, B., & Showers, N. (2014). Classrooms as Moderators of Developmental Education Effectiveness and Student Achievement. Community College Review, 42, 201-220.
  • Kelcey, B. & Swoboda, C. (2014). Prognostic Scores in Clustered Settings. In Propensity Score Analysis: Fundamentals, Developments, and Extensions Edited by Wei Pan & Haiyan Bai.
  • Kelcey, B., & Phelps, G. (2014). Strategies for Improving Power in School Randomized Studies of Professional Development. . Evaluation Review, 37, 520-554.
  • Kelcey, B., McGinn, D., & Hill, H. (2014). Approximate Measurement Invariance in Cross-classified Rater-mediated Assessments.. Frontiers in Quantitative Psychology and Measurement.
  • Carlisle, J., Kelcey, B., & Berebitsky, D. (2013). Teachers' Support of Students' Vocabulary Learning During Literacy Instruction in High Poverty Elementary Schools. American Educational Research Journal.
  • Kelcey, B., & Carlisle, J. (2013). Learning About Teachers’ Literacy Instruction From Classroom Observations. Reading Research Quarterly, 48, 301-317.
  • Kelcey, B., & Phelps, G. (2013). Considerations for Designing Group Randomized Trials of Professional Development with Teacher Knowledge Outcomes. Educational Evaluation and Policy Analysis, 35, 370-390.
  • Frank, K., Maurolis, S., Duong, M., & Kelcey, B. (2013). What would it take to Change an Inference?: Using Rubin’s Causal Model to Interpret the Robustness of Causal Inferences. Educational Evaluation and Policy Analysis, 35, 437-460.
  • Carlisle, J., Kelcey, B., Rosaen, C., Phelps, G., & Vereb, A. (2013). A Framework for Analysis of Case Studies of Reading Lessons. Journal of Education and Training Studies , 224-238.
  • Kelcey, B. (2011). Assessing the Effects of Teachers’ Reading Knowledge on Students’ Achievement Using Multilevel Propensity Score Stratification. Educational Evaluation and Policy Analysis, 33, 458-482.
  • Kelcey, B. (2011). Covariate Selection in Propensity Scores Using Outcome Proxies. Multivariate Behavioral Research, 46, 453-476.
  • Carlisle, J., Kelcey, B., Berebitsky, D., & Phelps, G. (2011). Embracing the Complexity of Instruction: A Study of the Effects of Teachers' Instruction on Students' Reading Comprehension. Scientific Studies of Reading, 15, 409-439.
  • Carlisle, J., Kelcey, B., Rowan, B., & Phelps, G. (2011). Teachers' Knowledge About Early Reading: Effects on Students' Gains in Reading Achievement. Journal for Research on Educational Effectiveness, 4, 289-321.
  • Songer, N. B., Kelcey, B., & Gotwals, A. (2009). Empirically Driven Development of a Learning Progression Focused in Complex Reasoning about Biodiversity. Journal of Research in Science Teaching, 46, 610-631.
  • Dong, N., Spybrook, J., & Kelcey, B. (in review) Power analyses for moderator effects in with (Non)Random Slopes in cluster randomized trials..
  • Cox, K. & Kelcey, B. (in review) Effects of Misspecified Parameter Values on Statistical Power in Optimally Designed Group-Randomized Studies of Mediation. Methodology.
  • Kelcey, B., Hill, H., & Chin, M. (in press) Teacher Mathematical Knowledge, Instructional Quality, and Student Outcomes: A Multilevel Mediation Analysis.. School Effectiveness & School Improvement.
  • Kelcey, B., Cox, K., & Dong, N. (in review) A Bias-Corrected Limited Information Estimator for Small to Moderate Scale Multilevel Structural Equation Models ..
  • Cox, K. & Kelcey, B. (in review) Effects of Misspecified Parameter Values on Statistical Power in Optimally Designed Group-Randomized Studies of Mediation..
  • Cox, K. & Kelcey, B. (in review) Statistical Power and Optimal Design for Multisite and Cluster-Randomized Studies when the Outcome is Measured with Error..
  • Cox, K. & Kelcey, B. (in review) A Partial Posterior Predictive Distribution Test for Multilevel Mediation..
  • Jensen, B., LeBaron, T., Covay-Minor, E., Davis, D., Dietiker, L., Gabriel, R., Halpin, P., Kelcey, B., Rui, N., & Steinberg, M. (in review) Complexity and Scale in Research on Teaching Effectiveness..
  • Spybrook, J., Kelcey, B., & Dong, N. (in review) Statistical Power in Multilevel Models. In Multilevel Modeling Methods with Introductory and Advanced Applications edited by Ann O’Connell, Betsy McCoach, Bethany Bell..

Presentations & Lectures

Paper Presentations

  • Kelcey, B. (2014). Noninvariant Measurement in Rater-Mediated Assessments of Teaching Quality.
  • Kelcey, B. (2014). Power Analysis for Cross Level Mediation in Cluster Randomized Trials.
  • Kelcey, B. (2014). Approximate Measurement Invariance in Rater‐mediated Assessments: Measuring Teaching Quality with Classroom Observations.
  • Kelcey,B., Phelps, G., & Spybrook, J. (2014). School Randomized Designs of Professional Development using Teacher Knowledge Outcomes.
  • Kelcey, B. & Spybrook, J. (2014). Designing Adequately Powered Cluster Randomized Trials using Optimal Design Plus.
  • Spybrook, J. & Kelcey, B. (2014). Power Calculations for Binary Moderators in Cluster Randomized Trials.
  • Kelcey, B. (2014). Noninvariant Measurement in Rater-mediated Assessments .
  • Kelcey, B. McGinn, D., Hill, H., & Charalambous, C. (2014). Dimensionality and Generalizability of the Mathematical Quality of Instruction Instrument.
  • Kelcey, B., & Carlisle, J. (2014). Teachers’ Knowledge about Teaching Reading, Their Instructional Practices, and Their Students’ Reading Achievement: Evidence from Quantile Mediation.
  • Carlisle, J., Connor, C., Kelcey, B., Petscher,Y., Sparapani, N., Ingebrand, S. (2014). Development of an Observation System: Creating Opportunities to Learn From Text.
  • Kelcey, B., Frank, K., & Seltzer, M. (2014). Parametric Sensitivity Analysis Thresholds for Mediation Effects.
  • McGinn, D., Kelcey, B., Hill, H., & Chin, M. (2014). Using Item Response Theory to Learn about Observational Instruments.
  • Kelcey, B. (2014). Multilevel Mediation: Design and Analysis.
  • Shen, Z. & Kelcey, B. (2014). Power Analysis for Multilevel Mediation.
  • Swoboda, C., Kelcey, B., & Zhang, J. (2014). Estimation of cross-classified random effects with missing data using multiple imputation.
  • Shen, Z. & Kelcey, B. (2014). Power Analysis for Cross-level Mediation.
  • Kelcey, B. & Phelps, G. (2013). Parameters for the Design of Group Randomized Studies for Teacher Professional Development.
  • Kelcey, B. McGinn, D., & Hill, H. (2013). Approximate Measurement Invariance in Rater-mediated Assessments: A Random Item Effects Item Response Model for Measuring Teaching Quality with Classroom Observations.
  • Kelcey, B. McGinn, D., & Hill, H. (2013). Measurement of Classroom Teaching Quality with Item Response Theory.
  • Kelcey, B. McGinn, D., & Hill, H. (2013). Measurement Issues in Teacher Observations and the Estimation of Teacher Effects.
  • Spybrook, J., Westine, C., Taylor, J., & Kelcey, B. (2013). Designing Adequately Powered Cluster Randomized Trials Using Optimal Design Plus.
  • Kelcey, B. (2013). Introduction to the Measures of Effective Teaching Longitudinal Database.
  • Kelcey, B. (2013). Psychometric Models for Measuring Individualized Instruction.
  • Kelcey, B. (2013). Propensity Score Matching within Prognostic Strata.
  • Kelcey, B. (2012). Propensity Score Matching within Prognostic Strata.
  • Kelcey, B., & Carlisle, J. (2012). Patterns of Instruction in Early Reading Lessons.
  • Kelcey, B., & Carlisle, J. (2012). A Multilevel Bifactor Framework for the Measurement of Multidimensional Instruction.
  • Frank, K., Maurolis, S., Duong, M., & Kelcey, B. (2012). A Quality Threshold for Statistical Controls in Observational Studies.
  • Moss, B., Kelcey, B., & Showers, N. (2012). Classrooms as Moderators of Developmental Education Effectiveness and Student Achievement.
  • Kelcey, B. (2012). A Multilevel Bifactor Framework for the Measurement of Teaching Across Subject Areas.
  • Kelcey, B. (2011). Inferences on Instructional Practice Measured With Multiple Sources of Variation.
  • Kelcey, B. (2011). Measurement with Cross-classified Item Response Models.
  • Kelcey, B. (2011). The Threshold of Embedded M Collider Bias and Confounding Bias.
  • Kelcey, B. (2011). Matching Across Schools with Multilevel Propensity Scores.
  • Gersh, A. & Kelcey, B. (2011). Improving New Teacher Effectiveness: An Analysis of Collective Teacher Knowledge and Possible Contextual Effects.
  • Carlisle, J., Kelcey, B., & Berebitsky, D. (2011). Teachers' Vocabulary Instruction and Word Meaning Interactions in High Poverty Schools.
  • Kelcey, B. (2010). Applications of Multilevel Propensity Scores.
  • Kelcey, B. (2010). A Multivariate, Multilevel Rasch Model for Characterizing Observed Instruction.
  • Kelcey, B., & Carlisle, J. (2010). A Triangulation of Teachers' Effectiveness via Value Added Models, Teacher Knowledge and Observed Instruction.
  • Kelcey, B. (2010). A Multivariate, Multilevel Rasch Model for Characterizing Teaching and Learning.
  • Kelcey, B. (2009). Propensity Score Variable Selection and Model Type in Multilevel Settings in which Treatment Assignment Varies by Schools.
  • Carlisle, J., Kelcey, B., Phelps, G., Johnson, D., & Berebitsky, D. (2009). Do teachers’ instructional practices during reading comprehension lessons contribute to their students’ progress in reading?.
  • Frank, K., Duong, M., Maurolis, S., & Kelcey, B. (2009). The Robustness of Inferences from Quasi- and Randomized Experiments.
  • Songer, N. B., Kelcey, B., & Gotwals, A. (2009). How and When Does Complex Reasoning Occur? Empirically Driven Development of a Learning Progression Focused on Complex Reasoning about Biodiversity.
  • Swoboda, C. & Kelcey, B. (2014). Combining propensity and prognostic scores in clustered settings.
  • Kelcey, B. (2015). Degenerate Power in Multilevel Mediation: The Non-monotonic Relationship Between Power & Effect Size.
  • Kelcey, B., Spybrook, J., Zhang, J., Phelps, G., & Jones, N. (2015). Strategies for Improving Power in Cluster Randomized Studies of Professional Development.

Creative Works

Design Work

  • PowerUpR Shiny App for Designing Multilevel Studies Probing Mediation . [Link]
  • PowerUpR Shiny App for Designing Multilevel Studies. [Link]