Evaluation is defined broadly in literature as the assessment of a system’s operation and output based upon a set of standards, where evaluators use the information to improve aspects of the system (Rossi, Lipsey, & Freeman, 2003; Weiss, 1998). Program evaluation can be used to ensure its components are aligned to standards, that they are fully implemented, and that they are having the desired effects (Bickman, 1987; Shadish, Cook, & Leviton, 1991). Evaluation of gifted education programs has assumed a greater role at the local, state, and national levels with the passage of legislation. For example, in the new Every Student Succeeds Act, states must include information about students’ achievement at advanced levels and how they plan to improve the skills of teachers and other school leaders in identifying and serving gifted and talented students. In the state of Texas, the Texas State Plan for the Education of Gifted/Talented Students provides a set of standards to assess the effectiveness of gifted services and ensures that the needs of gifted and talented students are being met.
The Texas State Board of Education (SBOE) adopted the Texas State Plan for the Education of Gifted/Talented Students (Texas State Plan) in 1989, which requires yearly district and campus-level evaluations where services are given a performance rating of “in compliance, recommended, exemplary” based on a set of standards (see 2.6C, 2.6.1R, 2.6E, 2.6.2R, 3.4.1R, 4.4C, 5.3C). The Texas State Plan delineates program compliance with state law in five categories of service that include student assessment, service design, professional development of personnel, curriculum and instruction, and family/community involvement. Using the Texas State Plan, national evaluation standards, and technically sound instruments, stakeholders are able to support and communicate the merit of gifted services to policymakers and ensure that gifted students are being served effectively.
To better understand evaluation tools and approaches, this review included articles that have been published since 2006 in Gifted Child Today, Gifted Child Quarterly, Journal for the Education of the Gifted, Journal of Advanced Academics, and Roeper Review. Articles needed to address evaluation of gifted education programming and services. Articles from other countries and those whose focus was not primarily about program evaluation were excluded. Using these criteria, 19 articles were identified and summarized. Of the articles included, two employed experimental methods to review evaluation (Cotabish & Robinson, 2012; Robinson, Cotabish, Wood, & O’Tuel, 2014), seven used mixed-methods designs (Hong, Greene, & Higgins, 2006; Matthews & Shaunessy, 2010; Peters & Gates, 2010; VanTassel-Baska, 2006; VanTassel-Baska, Quek, & Feng, 2006; Walker & VanderPloeg, 2015; Warne, 2014), three reviewed research completed (Kettler, 2016; Makel & Wai, 2016; Missett & Foster, 2015), and seven expressed opinions of the authors (Gallagher, 2006; McCoach, Rambo, & Welsh, 2013; Ryser & Rambo-Hernandez, 2014; Sanderson & Greenberger, 2011; Subotnik, Almarode, & Lee, 2016; Swanson & Lord, 2013; Welsh, 2011). Participants who were involved in the evaluation studies included school districts (Matthews & Shaunessy, 2010; VanTassel-Baska, 2006), G/T administrators (Cotabish & Robinson, 2012; Robinson et al., 2014), G/T and general education teachers (Hong et al., 2006; Peters & Gates, 2010; VanTassel-Baska et al., 2006) and students (Walker & VanderPloeg, 2015; Warne, 2014).
Use of Standards to Develop Evaluation
The NAGC Pre-K-Grade 12 Gifted Program Standards at the national level, and state-level standards like the Texas State Plan, are valuable tools in gifted program accountability and are meant to serve as references for impactful program evaluation, but translation from standards into practice can be a difficult process. Matthews and Shaunessy (2010) developed a programming checklist based upon the NAGC Gifted Program Standards and used the tool to evaluate a number of local programs. In doing so, the authors identified strengths of the national standards and areas that were challenging for programs to translate and implement (Matthews & Shaunessy, 2010). The authors concluded that while standards are necessary to develop and improve programs, the standards at the state and national levels must be user-friendly for practitioners. Gifted program coordinators and other stakeholders are able to evaluate their programs when program goals are operationalized, components of effective evaluation are addressed, reliable evaluation tools are used, and evaluation results are interpreted.
Components of Quality Evaluation
Rigorous evaluation models are needed within and across gifted education programs to examine program impact (Makel & Wai, 2016). Effective evaluations have common thematic components. Seven of the articles in this review discussed what makes a quality evaluation for different types of programs, including selective science, technology, engineering, and mathematics (STEM) school programs (Subotnik et al., 2016); online programs (Sanderson & Greenberger, 2011); and gifted programming as a whole (Kettler, 2016; Makel & Wai, 2016; Missett & Foster, 2015; Robinson et al., 2014; Walker & VanderPloeg, 2015). Because knowledge about best practices and research in gifted education is key to a quality evaluation, any evaluations conducted by those unfamiliar with the field should be validated by individuals who are experts in gifted and talented education (Kettler, 2016; Makel & Wai, 2016).
The purpose of the evaluation should be considered as a first step in the process. Evaluation can either be summative or formative (Gallagher, 2006). Summative evaluation involves collecting information for use outside of the program, and formative evaluation is intended for use by program staff to assess program efficiency and direct internal alterations (Gallagher, 2006). Treatment fidelity is key in making decisions from the evaluation results (Gallagher, 2006; Kettler, 2016; Missett & Foster, 2015). By identifying if gifted programs and services are implemented as designed, evaluators can ensure that assessments actually reflect the programs’ quality and their effects.
Gifted programs need to have clear program goals and descriptions (Robinson et al., 2014) with evaluations measuring the progress of outcomes toward established goals. Instructional approaches, learning objectives, and other elements should align with the program goals and priorities (Sanderson & Greenberger, 2011; Subotnik et al., 2016), and data collected for evaluation need to encompass key program and curricular components (Walker & VanderPloeg, 2015). Evaluation questions need to take into account individual program needs, including the demographic makeup of the population being served, the intervention curriculum, and whether the curriculum is meeting student needs (Kettler, 2016; Robinson et al., 2014; Sanderson & Greenberger, 2011). When evaluations are tailored to assess specific program characteristics, weaknesses and strengths can be identified and used to form better solutions for continued program improvement.
Instruments Used in Evaluation
Collection of data on program effectiveness, whether data are collected at the student, teacher, administrative, or program-wide level, is critical in allocating program resources (VanTassel-Baska, 2006). In order to make quality decisions based upon results of an evaluation, the instruments chosen for assessment must have reliability and be valid for the intended purpose, such as aligning with gifted program criteria (VanTassel-Baska, 2006). Six of the articles chosen for this review discussed tools to consider during various stages of the evaluation process (Hong et al., 2006; Peters & Gates, 2010; VanTassel-Baska et al., 2006; Welsh, 2011), and two articles proposed specific statistical approaches for longitudinal evaluation (McCoach et al., 2013, Ryser & Rambo-Hernandez, 2014).
Historically, gifted students have been a difficult population to monitor longitudinally because tests tend to measure proficiency only, making evaluation of a gifted program’s impact on student growth even more challenging. Three articles discussed a specific evaluation tool and statistical approach for assessing growth of gifted students. Warne (2014) proposed using above-level testing as a reliable tool for gifted student progress monitoring because these measures have higher ceilings and are more sensitive to the growth of higher ability students. In addition to using above-level testing, growth modeling should be considered when assessing academic change over time (McCoach et al., 2013, Ryser & Rambo-Hernandez, 2014). Rather than taking multiple snapshots of student performance measures, growth curve models (GCMs) can better represent student progress longitudinally, giving a better picture of program influence on student achievement.
Evaluation of teacher instructional practices is an essential component of ensuring program implementation fidelity and in determining if professional development is effective. Three articles focused on tools to evaluate teacher pedagogical practices and alignment with important components of gifted curriculum. The Instructional Practice Questionnaire (IPQ, Hong et al., 2006), the Differentiation Classroom Observation Scale (COS-R, VanTassel-Baska et al., 2006), and the Teacher Observation Form (TOF, Peters & Gates, 2010) were introduced as reliable measures to assess teachers’ use of differentiation techniques and other important best practice behaviors. Although teachers’ implementation of instructional practices are important to consider as part of the evaluation process, teachers can use best practice techniques and still have poor student outcomes due to other sources of variability. For example, other influences that impact teacher success include the curriculum and how it is implemented, student aptitude, classroom climate, teacher
student rapport, and so on. One of the included articles took into consideration teacher effectiveness as an independent factor. Welsh (2011) proposed using simulation-based effectiveness measures (SBEMs) or virtual techniques that create artificial classroom environments to eliminate extraneous sources of error and better evaluate teacher effects.
Teaching Evaluation Skills
Two articles addressed the importance of teaching evaluation skills. Peer coaching (Cotabish & Robinson, 2012) and participation in professional development (Robinson et al., 2014) were found to increase administrators’ evaluation knowledge and skills. Valuable thematic areas of development that were identified in both studies were administrators’ ability to prioritize evaluation questions, collect data to answer the evaluation questions, interpret evaluation results, and communicate results with stakeholders (Cotabish & Robinson, 2012; Robinson et al., 2014).
Interpretations of Evaluation
Stakeholders in gifted education programming, including economic policymakers, have interests in whether students are benefitting from services. After evidence of program impact and function has been collected, the next step in the evaluation process is the interpretation of results for translation into practice. Three of the articles suggested caution when developing conclusions (Gallagher, 2006), particularly without the aid of experts in gifted education who are familiar with programming priorities and evaluation nuances (Kettler, 2016; Makel & Wai, 2016). Errors in interpretation can lead to premature alterations, to a failure in justifying the need for the program, and even to the elimination of services. For this reason, evaluation research standards must be rigorous and interpretation must be held to a high standard (Makel & Wai, 2016).
Effects of Evaluation and Conclusion
Treatment drift is the tendency for individuals to shift over time toward practices that are comfortable and familiar, despite education about importance of a new practice or technique (Gallagher, 2006). It is because of this tendency to regress at an individual or systemic level that evaluation must be an ongoing process. Whether an evaluation is intended to assess an external product or internal mechanism, valid evidence must drive
decision-making. It is the responsibility of educators to purposefully track data and policy influence on programming (Swanson & Lord, 2013). Experts in gifted education are obligated to contribute to the evaluation process and interpretation of results for informed decision-making (Kettler, 2016; Makel & Wai, 2016). With these combined efforts, it is possible to provide strong evidence to gain support from policy makers and mobilize resources (Subotnik et al., 2016; VanTassel-Baska, 2006). Staying current on research and best practices for evaluation encourages forward movement toward established benchmarks and improves services to meet gifted and talented student needs.
Literature Review
Cotabish, A., & Robinson, A. (2012). The effects of peer coaching on the evaluation knowledge, skills, and concerns of gifted program administrators. Gifted Child Quarterly, 56, 160–170.
The purpose of this study was to assess the effects of peer coaching on evaluation knowledge and skills. The chosen participants (n = 99) received professional development focused on program evaluation from one of two institutes. Thirty were randomly selected to participate in the peer coaching condition and received intensive one-on-one peer coaching; the remaining 69 participants acted as a control group and didn’t receive additional intervention. Peer coaching involved the administrators’ review of the 1998 NAGC Pre-K-Grade 12 Gifted Program Standards, followed by rating themselves and their district based upon the criteria from the standards, setting goals that were a priority for the administrator’s campus, and monitoring progress with the peer coach during the following year of intervention. Peer coach services varied depending on administrator need, which included goals related to underidentification of students, service delivery to minority and low-income students, advocacy for student needs to other district staff, creation of stakeholder surveys to gauge involvement, and construction of district acceleration policies. Assessment of the effect of peer coaching was completed using the Change Facilitator Stages of Concern Questionnaire (CFSoCQ), which addressed stages of concern about conducting a gifted program evaluation. Peer coaching was found to be more effective at increasing administrators’ knowledge and skills regarding program evaluation. Essential components involved knowledge about prioritizing evaluation questions, collecting data to answer the evaluation questions, and interpreting and communicating evaluation results.
Gallagher, J. J. (2006). According to Jim Gallagher: How to shoot oneself in the foot with program evaluation. Roeper Review, 28, 122–124.
The author discussed the unintended consequences of errors in the evaluation process where lack of evidence of program effectiveness can lead to premature change or elimination of the program. Gallagher differentiated between summative evaluation, where data are collected for use outside of the program itself with an emphasis on a product, and formative evaluation, where a program’s staff uses data to examine program efficiency and the need for procedural changes. Gallagher suggested that asking the right questions during the evaluation process is essential, and maintaining progress goals develops a positive force for change with the information collected. Gallagher also emphasized the need to look at multiple components, including assessment of the fidelity of the original program’s implementation, effects of the administered program, and examination of how the program can be changed to increase effectiveness. In a classroom, teachers must be trained to use a teaching method and must be able to implement it correctly en vivo. Decisions about the teaching method’s effect on student performance cannot be made until the teacher is able to use the method consistently. Treatment drift can also cause underestimation of treatment effects, wherein a teacher may learn a method and be able to implement it correctly, but over time can unintentionally drift back to a more comfortable teaching method.
Hong, E., Greene, M. T., & Higgins, K. (2006). Instructional practices of teachers in general education classrooms and gifted resource rooms: Development and validation of the Instructional Practice Questionnaire. Gifted Child Quarterly, 50, 91–103.
In this article, the authors discussed the development of the Instructional Practice Questionnaire (IPQ), an instrument that consists of 28 items intended to assess how well a teacher differentiates learning experiences for learners in general education and gifted resource classrooms. The instrument addresses the cognitive, interpersonal, and intrapersonal needs of students during instruction. In this study, the instrument was piloted with 211 teachers from the third through fifth grade. To determine content validity, experts were consulted regarding item development and inclusion. Data were analyzed to determine the instrument’s internal consistency reliability. Alpha coefficients ranged from .77 to .90. Factor analysis supported the three domains addressed by the instrument: cognitive, interpersonal, and intrapersonal. Using the instrument, gifted education teachers reported that they engaged in differentiated instruction more frequently than the general education teachers. The authors stated that the IPQ may fill the need for an instrument that effectively evaluates instructional differentiation and practices in more than one domain.
Kettler, T. (2016). Why are economists evaluating the impact of gifted education? Journal of Advanced Academics, 27, 81–89.
In this article, Kettler responded to an article by economists (Bui, Craig, & Imberman, 2014) who suggested that NCLB has not had any negative impact on top students despite the diverting of educational resources toward the struggling student population and away from gifted education. Kettler pointed out several flaws in their study: (a) characteristics of students that were included in the study did not necessarily match typical conceptions of giftedness; (b) the outcome measure might not have been aligned to the program; and (c) interpretations of the data by economic policymakers might have led to faulty assertion that gifted education does not impact academic achievement. Kettler encouraged researchers and educators to participate in effective program evaluation for better assessment of the benefits and effectiveness of gifted programs. He suggested quality program evaluation includes four components: a description of the group demographics, the intervention program and curriculum, the fidelity of implementation, and the capacity of implementers. Those who are outside and unfamiliar with the field cannot be left to drive policy-making decisions based on evaluations that are not validated by experts in the field.
Makel, M. C., & Wai, J. (2016). Does economic research in education work? For which studies? Journal of Advanced Academics, 27, 73–80.
The authors discussed a study conducted by Card and Giuliano (2014) and described what it did well, how it could have been improved, and what gifted educators can learn from economists’ evaluation of gifted programs. In identifying what they did well, Makel and Wai described how Card and Giuliano used multiple analyses to assess the effects of a gifted education program and the impact of different practices used to identify students. Similar to this study, the authors suggested that evaluators consider rigorous research models that include variations within and across gifted education programs to examine effects. These variations might include identification methods, qualification criteria, program model, and duration of the interventions. The authors also felt that Card and Giuliano should have framed their evaluation question to indicate that they were focusing on an individual program rather than gifted education in general. Generalization should occur only when programs use similar identifications and interventions with similar groups. They suggested that the authors disregarded alternative interpretations that might have included the purpose of the gifted program, differential teacher attention, ceiling effects of achievement tests, and the existing research literature in gifted education.
Matthews, M. S., & Shaunessy, E. (2010). Putting standards into practice: Evaluating the utility of the NAGC Pre-K-Grade 12 Gifted Program Standards. Gifted Child Quarterly, 54, 159–167.
Through qualitative analysis of the “minimum” and “exemplary” criteria from the 1998 NAGC Pre-K–Grade 12 Gifted Program Standards, a 27-item checklist was developed to assess program quality and then piloted in the evaluation of 43 local plans for identification of diverse gifted learners. Strengths of the NAGC Gifted Program Standards were discussed and suggestions were made for revision to make them more user-friendly. The authors identified three broad areas of difficulty during checklist creation: (a) qualification of a district as “exemplary” for a standard but simultaneously not meeting the “minimum” standard, (b) inclusion of multiple criteria under one standard instead of separating goals, and (c) the use of nonspecific language or expectations that had not been operationalized.
McCoach, D. B., Rambo, K. E., & Welsh, M. (2013). Assessing the growth of gifted students. Gifted Child Quarterly, 57, 56–67.
In this article, the authors discussed historical difficulties in the measurement and tracking of gifted student growth and proposed the use of growth modeling for assessment of academic change over time for high-ability learners as an alternative. In the past, accountability measures focused on overall academic achievement of a school, comparing multiple snapshots of student performance on measures like state assessments to evaluate student and schoolwide change. McCoach and colleagues suggested using growth curve models (GCMs) to assess both intrapersonal and interpersonal change of all students but emphasized the approach’s utility in tracking gifted students. The authors specified three conditions in the creation of GCMs, which included (a) collection of data from the same individuals across multiple time points, (b) accurate measure of the time interval between measures, and (c) use of parallel or comparable measures across each time point. Use of measures with high ceilings and adequate content coverage is also important to consider when assessing gifted student progress.
Missett, T. C., & Foster, L. H. (2015). Searching for evidence-based practice: A survey of empirical studies on curricular interventions measuring and reporting fidelity of implementation published during 2004–2013. Journal of Advanced Academics, 26, 96–111.
Because Fidelity of Implementation (FOI) is such an important component in effective service delivery and modification, the authors sought to evaluate literature in the field of gifted education to determine the extent to which FOI is assessed, reported, and associated with outcome evaluation. Results from the survey indicated that FOI is becoming increasingly prioritized in gifted literature and researchers often recognize the importance of FOI; however, FOI is still represented very broadly in the evaluation of teaching practices and more systematic measurement of FOI is needed. When conducting treatment or program evaluation, it is essential that evaluators collect FOI data and determine its relationship to outcomes to make informed, valid decisions.
Peters, S. J., & Gates, J. C. (2010). The teacher observation form: Revisions and updates. Gifted Child Quarterly, 54, 179–188.
This article discussed the development and revisions made to the Teacher Observation Form (TOF), along with its usefulness as an evaluation tool for progress monitoring and surety of effective pedagogical practice. The TOF form was originally developed to evaluate teachers after completion of their gifted education practicum experience to provide feedback on performance. In the last 25 years, district administrators and other professionals have used the TOF more broadly to inform professional development and provide objective feedback for improvement of instructional practices. For this study, experts were contacted and asked to evaluate TOF items on a 5-point Likert-type scale to determine its utility for evaluating teachers of gifted students. Item content, discrimination, and coverage were assessed and used to revise the TOF. The new instrument was then piloted over a 2-year period with pre-K–12 teachers in one university-based enrichment program. Data were analyzed to determine internal consistency and other descriptive statistics. Overall alpha reliability was .95; no interrater reliability data were collected. The authors suggested that more than one observer should collect multiple observations to ensure that a single evaluator or a single instance does not influence final results. Results indicated that the TOF items are clear and concise and assess a common set of practices important to gifted education.
Robinson, A., Cotabish, A., Wood, B. K., & O’Tuel, F. S. (2014). The effects of a statewide evaluation initiative in gifted education on practitioner knowledge, concerns, and program documentation. Journal of Advanced Academics, 25, 349–383.
The impact of the Arkansas Evaluation Initiative (AEI) in gifted education was assessed to determine its utility in improving district capacity to conduct formative program evaluations. The original goal of the AEI professional development was to provide practitioners with foundational knowledge regarding program evaluation and skills needed to conduct effective evaluations on their respective campuses with an emphasis on the following: (a) writing program descriptions, (b) developing program goals, (c) analyzing components of an evaluation, (d) developing evaluation questions, and (e) conducting focus groups. This study used a randomized design stratified by size of district to divide 200 gifted and talented coordinators into two treatment groups with the first assessing participants in AEI for the first two school years and the second group initially acting as a control group but then receiving the same treatment afterward. Practitioners who attended the AEI training demonstrated significantly more knowledge and skills in evaluation than the control group who did not receive the AEI training. The experimental group also was better able to define evaluation, describe their program goals, form evaluation questions, articulate priorities for program revision, articulate essential components of a focus group, and link evaluation questions to standards. The authors concluded that program evaluation should be personalized for coordinators and address affective and attitudinal issues about evaluation.
Ryser, G. R., & Rambo-Hernandez, K. E. (2014). Using growth models to measure school performance: Implications for gifted learners. Gifted Child Today, 37, 17–23.
Instead of assessing student proficiency at one point in time, these researchers proposed that growth models better gauge Adequate Yearly Progress (AYP) and track student progress over time. Under No Child Left Behind (NCLB), the primary purpose for annual assessments was to determine how many students below proficiency were showing growth, rather than tracking growth of gifted students who typically score above proficiency on state assessments. The authors emphasized the importance of including at least three comparable observations or test scores in the assessment model to show growth of gifted students. Other components to consider when assessing gifted student growth are using above-level tests to avoid ceiling effects and vertically scaled assessments and regression toward the mean for extreme scores. Once these conditions have been taken into account, growth modeling can be a useful tool in progress monitoring of gifted students. However, caution needs to be used in relating growth scores to teacher evaluations because most tests have fewer items in the upper ranges, and gifted and talented students may show less growth than typical or average students.
Sanderson, E., & Greenberger, R. (2011). Evaluating online programs through a gifted lens. Gifted Child Today, 34(3), 42–55.
The authors suggested that online learning provides access to equitable education for all students and is a promising tool offering greater differentiation to gifted students who need more rigorous and challenging curriculum. The authors emphasized the importance of understanding what represents a quality online program by examining standards established by the International Association for K–12 Online Learning (iNACOL), Southern Regional Education Board (SREB), the National Education Association (NEA), and the National Association for Gifted Children (NAGC). Important characteristics include quality of instruction; appropriate learning objectives; and integration of problem solving, creativity, and metacognitive skills within learning activities. Evaluating online learning opportunities should also include questions related to how the program will meet the needs of individual students.
Subotnik, R. F., Almarode, J., & Lee, G. M. (2016). STEM schools as incubators of talent development. Gifted Child Today, 39, 236–241.
This article focused on the mission of selective science, technology, engineering, and mathematics (STEM) schools and how school goals must be compared to student outcomes to assess effectiveness, particularly if justification is needed for policymaker support. Selection of students for admission who are able to meet school and curricular goals is necessary, but the authors emphasized a need for appropriate vetting processes for entry. After describing the vision, selection, and instructional strategies for residential, comprehensive, school-within-a-school, shared-day, and early college STEM options, the authors described data that each school collected to evaluate its effectiveness. They concluded that having strong evidence that supports the school’s success is important to gaining support from policy makers.
Swanson, J. D., & Lord, E. W. (2013). Harnessing and guiding the power of policy: Examples from one state’s experiences. Journal for the Education of the Gifted, 36, 198–219.
The authors discussed the role of identification, program management, assessment, evaluation, and other factors that contributed to gifted education policy development and included an illustrative case from the state of North Carolina to help readers understand the impact of policy on program success. The use of effective evaluation techniques that evidence program impact ensures continuous improvement of gifted programs through policy change. Swanson and Lord emphasized the importance of monitoring and collecting data on the changes incurred from policy adjustment in order to keep policy change for gifted education moving forward.
VanTassel-Baska, J. (2006). A content analysis of evaluation findings across 20 gifted programs: A clarion call for enhanced gifted program development. Gifted Child Quarterly, 50, 199–210.
The purpose of this study was to identify major themes across evaluations, document the process of synthesizing findings across multiple evaluation studies, and provide insights for gifted program improvement. The author analyzed seven gifted program evaluation studies representing 20 school districts. The evaluations examined student identification, curriculum, staff development, parental involvement, and evaluation. Schools that were included in the evaluations consisted of a relatively representative sample and used a range of service models for their gifted students. The evaluations were launched initially by collecting data from meetings with stakeholders to develop an evaluation plan. Following the development of the plan, quantitative data were collected through educator and parent surveys and classroom observations. Qualitative data were collected through document review, individual interviews, and focus group protocols. Common themes that emerged were dedicated personnel, perceived challenging student opportunities, diversity of approaches and options, lack of systematic evaluation, imperfect identification systems, program incoherence, limitations in personnel preparation, and a lack of adequate resources.
VanTassel‐Baska, J., Quek, C., & Feng, A. X. (2006). The development and use of a structured teacher observation scale to assess differentiated best practice. Roeper Review, 29, 84–92.
This article discussed the use of a classroom observation instrument, the Differentiation Classroom Observation Scale (COS-R) that assesses professional development needs of teachers of gifted students. The authors described how the COS-R might be used in examining important components of gifted education, which include curriculum planning and delivery, differentiated teaching behaviors, accommodation for individual differences, problem-solving strategies, critical thinking strategies, creative thinking strategies, and research strategies. The COS-R was piloted with 50 teachers who taught in a Saturday Enrichment Program at William & Mary. Analyses indicated that the COS-R was highly reliable (Alpha = .91 to .93) and also had strong interrater reliability (.87 and .89). The authors suggested that the COS-R provides a method for assessing and improving teaching performance and might be used to examine the influence of teaching on gifted students’ outcomes.
Walker, L. D., & VanderPloeg, M. K. (2015). Surveying graduates of a self-contained high school gifted program: A tool for evaluation, development, and strategic planning. Gifted Child Today, 38, 160–176.
The Commonwealth Governor’s School (CGS) is a high school program for academically gifted students and is operated by a consortium of counties in Virginia. The purpose of this qualitative and quantitative study was to describe the benefits of the CGS as perceived by its graduates. The survey consisted of two types of data with free-response type questions and Likert-type scale questions. Data were collected from 134 CGS graduates and 266 members of a comparison group, which were matched to the CGS graduates using class rank as a measure of achievement and SAT score as an indicator for aptitude. The response rate for the CGS group was 57.5% (n=77) versus 34% (n=94) for the comparison group. Results indicated that the CGS group strongly agreed that their high school program provided more challenge, taught more research skills, and developed more communication and presentation skills than the comparison group. Both the CGS and comparison groups strongly agreed that their high school program provided mentoring and faculty support.
Warne, R. T. (2014). Using above-level testing to track growth in academic achievement in gifted students. Gifted Child Quarterly, 58, 3–23.
In this article, Warne provided support for the use of above-level testing for gifted student progress monitoring. The participants consisted of 222 gifted middle school students who were selected from a magnet program and divided into four groups based upon grade level. The Iowa Tests of Basic Skills (ITBS) and the Iowa Tests of Educational Development (ITED) were used to evaluate student performance at multiple points in time across a 2-year period. Despite ethnicity having a significant impact on performance between groups, results indicated that the administration of the ITBS and ITED that were above-level or 2 years higher than the age of participants was supported by reliability evidence. Above-level assessments may provide a higher ceiling and be effective tools for measuring high-ability student performance. Above-level test scores were found to be effective in tracking individual progress and should be considered in evaluating gifted student growth and gifted program effectiveness.
Welsh, M. E. (2011). Measuring teacher effectiveness in gifted education: Some challenges and suggestions. Journal of Advanced Academics, 22, 750–770.
In this article, Welsh proposed the use of simulation-based effectiveness measures (SBEMs) to combat weaknesses that are innate in evaluating teacher effectiveness, particularly with teachers of gifted students where student growth is difficult to assess over time and effects are likely to be attenuated. In an educational context, SBEM’s can simulate and measure teacher effectiveness by standardizing student performance and content taught, allowing for direct comparison across professionals and providing valuable feedback for professional development. Teachers would complete the SBEM after reviewing the class information and preparing teaching materials. They would then enter the virtual reality classroom and teach their lesson with the simulated students. The SBEM would provide an alternative to traditional assessments of teacher effectiveness: observations, student achievement, and teacher tests.
Additional References
Bickman, L. (1987). The functions of program theory. New Directions for Evaluation, 1987(33), 5–18.
Rossi, P. H., Lipsey, M. W., & Freeman, H. E. (2003). Evaluation: A systematic approach. Thousand Oaks, CA: Sage.
Shadish, W. R., Cook, T. D., & Leviton, L. C. (1991). Foundations of program evaluation: Theories of practice. Thousand Oaks, CA: Sage.
Weiss, C. H. (1998). Have we learned anything new about the use of evaluation? The American Journal of Evaluation, 19(1), 21–33.
Rebecca J. Tipton received her bachelor’s degrees in psychology and biology and physiology and cell biology from Saint Louis University. She is currently a doctoral student in the Department of Educational Psychology at Baylor University where she is specializing in school psychology. Her research interests focus on gifted education, development and application of quantitative methodology, as well as students with exceptional learning needs, particularly twice-exceptional learners and at-risk students from low-income backgrounds. She may be reached at Rebecca_Tipton@baylor.edu
Susan K. Johnsen, Ph.D., is Professor Emeritus in the Department of Educational Psychology at Baylor University, where she directed programs related to gifted and talented education. She is the author of more than 250 publications including Identifying Gifted Students: A Practical Guide, books related to implementing the national teacher preparation standards in gifted education, and tests used in identifying gifted students. She is editor-in-chief of Gifted Child Today, past president of the Association for the Gifted (TAG), and past president of the Texas Association for the Gifted and Talented (TAGT). She has received awards for her work in the field of gifted education, including CEC’s Leadership Award, NAGC’s President’s Award, TAG’s Leadership Award, TAGT’s President’s Award, TAGT’s Advocacy Award, and Baylor University’s Investigator Award, Teaching Award, and Contributions to the Academic Community. She may be reached at Department of Educational Psychology, School of Education, Baylor University, One Bear Place #97301, Waco, TX 76798 or Susan_Johnsen@baylor.edu.
