Evaluation of treatment effect estimates in single-case experimental research: comparison of twelve overlap methods and visual analysis

Yucesoy-Ozkan Ş., Rakap S., Gulboy E.

BRITISH JOURNAL OF SPECIAL EDUCATION, cilt.47, sa.1, ss.67-87, 2020 (ESCI) identifier identifier

  • Yayın Türü: Makale / Tam Makale
  • Cilt numarası: 47 Sayı: 1
  • Basım Tarihi: 2020
  • Doi Numarası: 10.1111/1467-8578.12294
  • Dergi Adı: BRITISH JOURNAL OF SPECIAL EDUCATION
  • Derginin Tarandığı İndeksler: Emerging Sources Citation Index (ESCI), Scopus, Academic Search Premier, ASSIA, Child Development & Adolescent Studies, EBSCO Education Source, Educational research abstracts (ERA), ERIC (Education Resources Information Center), Linguistics & Language Behavior Abstracts, Psycinfo, DIALNET
  • Sayfa Sayıları: ss.67-87
  • Anahtar Kelimeler: single-case experimental research, visual analysis, treatment effect estimates, nonoverlap methods, effect size, QUANTITATIVE SYNTHESIS, SUBJECT RESEARCH, EFFECT SIZES, INTERVENTION, METAANALYSIS, PERCENTAGE, AGREEMENT
  • Eskişehir Osmangazi Üniversitesi Adresli: Hayır

Özet

The purpose of this study was to compare 12 commonly-used nonoverlap methods with each other and with the results of visual analysis. Data were obtained from 25 studies focused on embedded instruction and schema-based instruction and included a total of 101 graphs. Treatment effect estimates using 12 nonoverlap methods were calculated for each graph by hand or using an online calculator. Five experts conducted visual analysis of each graph. Results showed that strong agreements existed between visual analysis and PND, Tau(NOVLAP) and Tau-U when raw data were analysed, and PND, PNCD and PEM-T when categorised data were analysed. Of the 12 methods investigated, PND had the highest agreement rate with visual analysis, followed by PEM-T, PAND, PNCD, IRD, NAP and Tau(NOVLAP). Overall, visual analysis appeared to be more conservative, as most nonoverlap methods overestimated the treatment effect. Additional research is needed to replicate and potentially validate the findings of this study.