Our Faculty and Staff

< Back to list

Benjamin M Kelcey

Title: Associate Professor of Quantitative Research Methodologies
Office: 3311B TEACHER-DYER
Tel: 513-556-3608
Email: benjamin.kelcey@uc.edu

My research focuses on causal inference and measurement methods within the context of multilevel and multidimensional settings such as classrooms and schools. With respect to causal inference, I develop and apply methods for statistical adjustment that enable researchers to mount focused, specific analogies of their observational studies to randomized experiments with emphasis on multilevel or clustered settings. In this way, I view statistical methods not as substitutes for random assignment but rather as tools to uncover objective information that will lead to well specified and designed randomized studies. My measurement work is also generally situated within multilevel contexts (e.g., measurement of teaching) and develops and applies methods to address the multilevel and multidimensional nature of teaching and learning. Statistically, these research areas unfold within the context of multidimensional, multilevel and cross-classified explanatory item response models, mixed effect models, propensity score methods, and analogues and extensions of each of these. The confluence of these statistical foci take substantive root in understanding the extent to which teacher and school factors can intervene to promote more effective and contextually responsive classroom and school environments. This work has included the measurement and analysis of teachers' knowledge, instructional quality and strategies, and student achievement to identify and develop methods of effective teaching and to establish links among these constructs.

Education

  • Ph.D., University of Michigan, 2009 (Quantitative Research Methods).
  • M.A., University of Michigan, 2006 (Statistics).
  • B.S., University at Albany, 2001 (Mathematics).

Research Information

Research Support

  • Power Analyses for Moderator and Mediator Effects in Cluster Randomized Trials (with Jessaca Spybrook & Nianbo Dong), National Science Foundation. 2014 to 2017.
  • Kelcey, Benjamin, Multilevel Mediation Models to Study the Impact of Teacher Development on Student Achievement in Mathematics, National Science Foundation. 2016 to 2021.
  • Empirical Benchmarks of Design Parameters for Group Randomized Trials (with Geoffrey Phelps & Nathan Jones), National Science Foundation. 2012 to 2016.
  • Measuring Teaching with Cross-classified Random Item Effects Item Response Models, William T. Grant Foundation and the Spencer Foundation. 2013 to 2014.
  • Optimizing Learning Opportunities for Students' Early Learning Observation System (with Carol Connor), Institute of Education Sciences. 2016 to 2020.
  • Exploration of Writing Instruction for Kindergarten Children (with Ying Guo, Allison Breit-Smith, and Chris Swoboda), Institute of Education Sciences. 2016 to 2020.
  • Exploring Effective Reading Comprehension Instruction: Classroom Practice, Teacher, and Student Characteristics (with Carol Connor and Joanne Carlisle), Institute of Education Sciences. 2013 to 2016.
  • A data-driven approach to kindergarten readiness and the importance of the preschool years: A partnership between researchers, practitioners, and stakeholders (with Jessica Logan), Ohio Education Research Center. 2013 to 2014.
  • The Sensitivity of Classroom Observation Assessments of Teacher Quality to Scoring Model (with Heather Hill), National Center for Teacher Effectiveness. 2012 to 2013.
  • Multilevel Measurement Models for Understanding Teacher Quality (with Brian Rowan). 2011 to 2012.
  • Development of an Interactive, Multimedia Assessment of Teachers' Knowledge of Early Reading (with Joanne Carlisle), Institute of Education Sciences. 2010 to 2013.
  • Investigating the Measurement and Development of Mathematical Knowledge for Teaching (with Geoffrey Phelps), National Science Foundation. 2009 to 2012.
  • Assessment of Pedagogical Content Knowledge of Teachers of Reading (with Joanne Carlisle), Institute of Education Sciences. 2009 to 2011.
  • Mission Possible: Graudation (MPG) King-Chavez Parks (KCP) Initiative (with S. Kanoyton), State of Michigan Dept of Energy. 2009 to 2010.
  • Measuring Teacher Instruction (with S. Kanoyton), State of Michigan Dept of Energy. 2009 to 2010.
  • Kelcey, Benjamin, Empirical Benchmarks of Design Parameters for Group Randomized Trials , National Science Foundation. BAD DATE to 2016. Status: Active.
  • Kelcey, Benjamin, Measuring Teaching with Cross-Classified Random Item Effects Item Response Models, William T. Grant Foundation. 08/15/2013 to 07/31/2014.
  • Kelcey, Benjamin, Power Analyses for Moderator and Mediator Effects in Cluster Randomized Trials, National Science Foundation. 2014 to 2017. Status: Active.
  • The Day Reconstruction Method: A New Tool for Measuring the Work of K-12 Teachers, and the Contexts of Their Work (with E. Camburn), Institute of Education Sciences. 2016 to 2020.
  • ..., . ....

Publications

Peer-reviewed Publications

  • Kelcey,B. Dong, N., Spybrook, J., & Cox, K. (2017). Statistical Power for Causally-defined Indirect Effects in Group-randomized Trials with Individual-level Mediators . Journal of Educational and Behavioral Statistics.
  • Kelcey, B., & Shen, Z. (2016). Multilevel Design of School Effectiveness Studies in Sub-Saharan Africa. School Effectiveness and School Improvement.
  • Kelcey, B., Dong, N., Spybrook, J., & Shen, Z. (2017). Experimental Power for Indirect Effects in Group-randomized Studies with Group-level Mediators . Multivariate Behavioral Research.
  • Spybrook, J., Kelcey, B., & Dong, N. (2016). Power Analyses for Detecting Treatment by Moderator Effects in Cluster Randomized Trials. Journal of Educational and Behavioral Statistics.
  • Kelcey, B., McGinn, D., & Hill, H. (2014). Approximate Measurement Invariance in Cross-classified Rater-mediated Assessments.. Frontiers in Quantitative Psychology and Measurement.
  • Spybrook, J., Shi, R., & Kelcey, B. (2016). Progress in the Past Decade: An Examination of the Precision of Cluster Randomized Trials Funded by the U.S. Institute of Education Sciences. International Journal of Research & Method in Education.
  • Dwyer, J. Kelcey, B., Berebitsky, D., & Carlisle, J. (2016). A Study of Teachers’ Discourse Moves That Support Text-based Discussions. Elementary School Journal..
  • Adelson, J., & Kelcey, B. (2016). Bringing a perspective from outside the field: A commentary on Davis et al.’s (2010) use of a modified regression discontinuity design to evaluate a gifted program.. Journal of Advanced Academics.
  • Kelcey, B., & Phelps, G. (2014). Strategies for Improving Power in School Randomized Studies of Professional Development. . Evaluation Review, 37, 520-554.
  • Moss, B., Kelcey, B., & Showers, N. (2014). Classrooms as Moderators of Developmental Education Effectiveness and Student Achievement. Community College Review, 42, 201-220.
  • Kelcey, B., & Carlisle, J. (2013). Learning About Teachers’ Literacy Instruction From Classroom Observations. Reading Research Quarterly, 48, 301-317.
  • Carlisle, J., Kelcey, B., & Berebitsky, D. (2013). Teachers' Support of Students' Vocabulary Learning During Literacy Instruction in High Poverty Elementary Schools. American Educational Research Journal.
  • Kelcey, B., & Phelps, G. (2013). Considerations for Designing Group Randomized Trials of Professional Development with Teacher Knowledge Outcomes. Educational Evaluation and Policy Analysis, 35, 370-390.
  • Frank, K., Maurolis, S., Duong, M., & Kelcey, B. (2013). What would it take to Change an Inference?: Using Rubin’s Causal Model to Interpret the Robustness of Causal Inferences. Educational Evaluation and Policy Analysis, 35, 437-460.
  • Carlisle, J., Kelcey, B., Rosaen, C., Phelps, G., & Vereb, A. (2013). A Framework for Analysis of Case Studies of Reading Lessons. Journal of Education and Training Studies, 224-238.
  • Kelcey, B. (2011). Assessing the Effects of Teachers’ Reading Knowledge on Students’ Achievement Using Multilevel Propensity Score Stratification. Educational Evaluation and Policy Analysis, 33, 458-482.
  • Kelcey, B. (2011). Covariate Selection in Propensity Scores Using Outcome Proxies. Multivariate Behavioral Research, 46, 453-476.
  • Carlisle, J., Kelcey, B., Berebitsky, D., & Phelps, G. (2011). Embracing the Complexity of Instruction: A Study of the Effects of Teachers' Instruction on Students' Reading Comprehension. Scientific Studies of Reading, 15, 409-439.
  • Carlisle, J., Kelcey, B., Rowan, B., & Phelps, G. (2011). Teachers' Knowledge About Early Reading: Effects on Students' Gains in Reading Achievement. Journal For Research on Educational Effectiveness, 4, 289-321.
  • Songer, N. B., Kelcey, B., & Gotwals, A. (2009). Empirically Driven Development of a Learning Progression Focused in Complex Reasoning about Biodiversity. Journal of Research in Science Teaching, 46, 610-631.
  • Kelcey, B., & Swoboda, C. (2014). Prognostic Scores in Clustered Settings. In Propensity Score Analysis: Fundamentals, Developments, and Extensions Edited by Wei Pan & Haiyan Bai.
  • Zaidi, N., Swoboda, C., Kelcey, B., & Manuel, S. (2016). Hidden Item Variance in Multiple Mini-Interview Scores. Advances in Health Sciences Education.
  • Phelps, G., Kelcey, B., Liu, S., & Jones, N. (2016). Performance Trajectories of Teacher Knowledge Growth Throughout Professional Development Programs.. Evaluation Review.
  • Kelcey,B. (2014). Measuring Teachers’ Instruction with Multilevel Item Response Theory.. International Journal of Research in Education Methodology.
  • Kelcey, B, Shen, Z., Spybrook, J. (2016). Intraclass Correlation Coefficients for Designing School Randomized Trials in Education in Sub-Saharan Africa. Evaluation Review.
  • Shen, Z., Escue-Simon, C., & Kelcey, B. (2017). The potential consequences of using value-added models to evaluate teachers. EJournal of Education Policy.
  • Kelcey, B., Phelps, G., Spybrook, J., Jones, N., & Zhang, J. (2017). Designing Large-scale Multisite and Cluster-randomized Studies of Professional Development. Journal of Experimental Education.
  • Wallace, T., Kelcey, B., & Ruzek, E. (2016). What Can Student Perception Surveys Tell Us About Teaching?: Empirically Testing Theoretically-Proposed Dimensions of Teaching. American Educational Research Journal.
  • Spybrook, J., & Kelcey, B. (2016). Introduction to Three Special Issues on Design Parameters Values for Planning Cluster Randomized Trials in the Social Sciences. Evaluation Review, 40, 491-499.
  • Dong, N., Kelcey, B., & Spybrook, J. (2017). Power analyses for moderator effects in three-level cluster randomized trials. Journal of Experimental Education.
  • Dong, N., Spybrook, J., & Kelcey, B. (In Press). Power analyses for moderator effects in with (Non)Random Slopes in cluster randomized trials. Journal of Research on Educational Effectiveness.
  • Zaidi, N. L. B., Swoboda, C. M., Kelcey, B. M., & Manuel, R. S. (2017). Hidden item variance in multiple mini-interview scores.. Advances in Health Sciences Education : Theory and Practice, 22 (2), 337-363.

Presentations & Lectures

Paper Presentations

  • Kelcey, B. (2014). Noninvariant Measurement in Rater-Mediated Assessments of Teaching Quality. Society for Research on Educational Effectiveness.
  • Kelcey, B. (2014). Power Analysis for Cross Level Mediation in Cluster Randomized Trials. Society for Research on Educational Effectiveness.
  • Kelcey, B. (2014). Approximate Measurement Invariance in Rater‐mediated Assessments: Measuring Teaching Quality with Classroom Observations. Hawaii International Conference on Education.
  • Kelcey,B., Phelps, G., & Spybrook, J. (2014). School Randomized Designs of Professional Development using Teacher Knowledge Outcomes. Hawaii International Conference on Education.
  • Kelcey, B. & Spybrook, J. (2014). Designing Adequately Powered Cluster Randomized Trials using Optimal Design Plus. Hawaii International Conference on Education.
  • Spybrook, J. & Kelcey, B. (2014). Power Calculations for Binary Moderators in Cluster Randomized Trials. Society for Research on Educational Effectiveness.
  • Kelcey, B. (2014). Noninvariant Measurement in Rater-mediated Assessments . American Educational Research Association.
  • Kelcey, B. McGinn, D., Hill, H., & Charalambous, C. (2014). Dimensionality and Generalizability of the Mathematical Quality of Instruction Instrument. National Council on Measuement in Education.
  • Kelcey, B., & Carlisle, J. (2014). Teachers’ Knowledge about Teaching Reading, Their Instructional Practices, and Their Students’ Reading Achievement: Evidence from Quantile Mediation. Society for the Scientific Study of Reading.
  • Carlisle, J., Connor, C., Kelcey, B., Petscher,Y., Sparapani, N., Ingebrand, S. (2014). Development of an Observation System: Creating Opportunities to Learn From Text. Society for the Scientific Study of Reading Conference.
  • Kelcey, B., Frank, K., & Seltzer, M. (2014). Parametric Sensitivity Analysis Thresholds for Mediation Effects. Modern Modeling Methods.
  • McGinn, D., Kelcey, B., Hill, H., & Chin, M. (2014). Using Item Response Theory to Learn about Observational Instruments. National Council on Measuement in Education.
  • Kelcey, B. (2014). Multilevel Mediation: Design and Analysis. Social Science Methodology Symposium.
  • Shen, Z. & Kelcey, B. (2014). Power Analysis for Multilevel Mediation. Tristate Research Conference.
  • Swoboda, C., Kelcey, B., & Zhang, J. (2014). Estimation of cross-classified random effects with missing data using multiple imputation. American Educational Research Association.
  • Shen, Z. & Kelcey, B. (2014). Power Analysis for Cross-level Mediation. University of Cincinnati Graduate Student Research Conference.
  • Kelcey, B. & Phelps, G. (2013). Parameters for the Design of Group Randomized Studies for Teacher Professional Development. Society for Research on Educational Effectiveness.
  • Kelcey, B. McGinn, D., & Hill, H. (2013). Approximate Measurement Invariance in Rater-mediated Assessments: A Random Item Effects Item Response Model for Measuring Teaching Quality with Classroom Observations. Modern Modeling Methods.
  • Kelcey, B. McGinn, D., & Hill, H. (2013). Measurement of Classroom Teaching Quality with Item Response Theory. Society for Research on Educational Effectiveness.
  • Kelcey, B. McGinn, D., & Hill, H. (2013). Measurement Issues in Teacher Observations and the Estimation of Teacher Effects. National Council on Measuement in Education.
  • Spybrook, J., Westine, C., Taylor, J., & Kelcey, B. (2013). Designing Adequately Powered Cluster Randomized Trials Using Optimal Design Plus. American Educational Research Association.
  • Kelcey, B. (2013). Introduction to the Measures of Effective Teaching Longitudinal Database. American Educational Research Association.
  • Kelcey, B. (2013). Psychometric Models for Measuring Individualized Instruction. Arizona State University Learning Sciences Institute.
  • Kelcey, B. (2013). Propensity Score Matching within Prognostic Strata. Society for Research on Educational Effectiveness.
  • Kelcey, B. (2012). Propensity Score Matching within Prognostic Strata. American Educational Research Association.
  • Kelcey, B., & Carlisle, J. (2012). Patterns of Instruction in Early Reading Lessons. Society for the Scientific Study of Reading.
  • Kelcey, B., & Carlisle, J. (2012). A Multilevel Bifactor Framework for the Measurement of Multidimensional Instruction. Society for Research on Educational Effectiveness.
  • Frank, K., Maurolis, S., Duong, M., & Kelcey, B. (2012). A Quality Threshold for Statistical Controls in Observational Studies. Joint Statistical Meeting.
  • Moss, B., Kelcey, B., & Showers, N. (2012). Classrooms as Moderators of Developmental Education Effectiveness and Student Achievement. Annual Association for Institutional Research Forum.
  • Kelcey, B. (2012). A Multilevel Bifactor Framework for the Measurement of Teaching Across Subject Areas. Society for Research on Child Development & Developmental Methodology.
  • Kelcey, B. (2011). Inferences on Instructional Practice Measured With Multiple Sources of Variation. Modern Modeling Methods.
  • Kelcey, B. (2011). Measurement with Cross-classified Item Response Models. Multilevel Modeling Conference.
  • Kelcey, B. (2011). The Threshold of Embedded M Collider Bias and Confounding Bias. Society for Research on Educational Effectiveness.
  • Kelcey, B. (2011). Matching Across Schools with Multilevel Propensity Scores. American Educational Research Association.
  • Gersh, A. & Kelcey, B. (2011). Improving New Teacher Effectiveness: An Analysis of Collective Teacher Knowledge and Possible Contextual Effects. American Educational Research Association.
  • Carlisle, J., Kelcey, B., & Berebitsky, D. (2011). Teachers' Vocabulary Instruction and Word Meaning Interactions in High Poverty Schools. American Educational Research Association.
  • Kelcey, B. (2010). Applications of Multilevel Propensity Scores. American Educational Research Association.
  • Kelcey, B. (2010). A Multivariate, Multilevel Rasch Model for Characterizing Observed Instruction. American Educational Research Association.
  • Kelcey, B., & Carlisle, J. (2010). A Triangulation of Teachers' Effectiveness via Value Added Models, Teacher Knowledge and Observed Instruction. Institute for Education Sciences Conference.
  • Kelcey, B. (2010). A Multivariate, Multilevel Rasch Model for Characterizing Teaching and Learning. Society for Research on Educational Effectiveness.
  • Kelcey, B. (2009). Propensity Score Variable Selection and Model Type in Multilevel Settings in which Treatment Assignment Varies by Schools. American Educational Research Association.
  • Carlisle, J., Kelcey, B., Phelps, G., Johnson, D., & Berebitsky, D. (2009). Do teachers’ instructional practices during reading comprehension lessons contribute to their students’ progress in reading?. Society for the Scientific Study of Reading Conference.
  • Frank, K., Duong, M., Maurolis, S., & Kelcey, B. (2009). The Robustness of Inferences from Quasi- and Randomized Experiments. Society for Research on Educational Effectiveness.
  • Songer, N. B., Kelcey, B., & Gotwals, A. (2009). How and When Does Complex Reasoning Occur? Empirically Driven Development of a Learning Progression Focused on Complex Reasoning about Biodiversity. American Educational Research Association.
  • Swoboda, C. & Kelcey, B. (2014). Combining propensity and prognostic scores in clustered settings. Annual Meeting of Psychometric Society.
  • Kelcey, B. (2015). Degenerate Power in Multilevel Mediation: The Non-monotonic Relationship Between Power & Effect Size. Society for Research On Educational Effectiveness.
  • Kelcey, B., Spybrook, J., Zhang, J., Phelps, G., & Jones, N. (2015). Strategies for Improving Power in Cluster Randomized Studies of Professional Development. Society for Research On Educational Effectiveness.