From Foresight, Vol. 7, No. 2
published 2000
Editor’s Note: In 1999, the Kentucky Department of Education (KDE) launched the Kentucky Teacher Data System Improvement Project to develop a plan or process for resolving current gaps in data about Kentucky teachers. This article summarizes the full report of a Steering Committee formed by KDE to examine data issues and recommend a new data structure, which was approved in December 1999. The report was prepared by Dr. Stephen Clements, an assistant professor in the University of Kentucky's Department of Educational Policy Studies and Evaluation.
Arguably, the thrust of the 1990 Kentucky Education Reform Act (KERA) was to improve teaching and learning through an array of teacher improvement policies, but the early years of KERA were dominated by implementation challenges. Now, almost a decade after KERA’s passage, attention has begun to shift to teacher quality, in part as a result of a recent national focus on the issue. A number of efforts to assess teacher quality have been undertaken here, including a joint legislative/executive Task Force on Teacher Quality charged with recommending legislation for the 2000 session of the General Assembly. The Task Force became a focal point for recommendations from various private and public entities examining teacher quality. Legislation incorporating a few of these recommendations was debated in the 2000 session of the General Assembly. While the teacher quality bill that resulted did not pass intact, one thing became very clear: data about teachers in the Kentucky are deficient and cannot be used to answer basic policy questions on the minds of policymakers and the public.
Available data do provide basic facts about Kentucky’s estimated 40,000 teachers:
About three fourths are white females; the balance are mostly white males; minorities represent only 3.5 percent of the teacher workforce.
State teachers average 16 years of experience, but most are clustered at the low and high ends of experience levels.
Three fourths have a master’s degree or its equivalent, compared to national and regional averages of 47 percent.
Teacher training students in Kentucky average 21.5 on the ACT, above the state (20.1) and U.S. (20.9) average.
Recent teacher training graduates score right at the national averages on the general skills components of the Praxis II exams, but subject area scores vary.
1996-97 salaries were about $34,000, below the U.S. average of $38,500 but above the southeastern state average.
About one fourth of Kentucky’s teachers are within three years of retirement eligibility.
Teacher supply is greater here than in most states, but many rural districts have shortages, especially in high school math, chemistry, and physics, and in special education.
Beyond these and sundry other facts about teachers, we know little about what they know and can do. For example, we know little about how many middle and high school teachers are teaching subjects “out of field,” which some policymakers define as having neither a major nor minor. Under current certification rules, it is nearly impossible to be certified to teach a subject with less than a minor, but in the past it was possible. While KDE has all teacher transcripts and theoretically could match training with teaching assignments, the data are not available in electronic form. Teaching assignments can only be matched with certification records, and, as certification rules change, they often only apply to newly certified teachers.
Teacher supply and demand data are also sketchy at best and only permit gross generalizations about shortages or surpluses. We know how many teachers are close to retirement age and where they are, but we do not know when they will retire. Similarly, we can track teacher training students by their numbers, discipline areas, and projected graduation dates, but many will not teach here or perhaps not at all. And none of the extant teacher demand projection models incorporates information about demographic shifts in the state. Thus, policymakers cannot respond to potential shortages of qualified teachers, a problem that may worsen as a wave of veteran teachers retire.
Similarly, we have almost no data on the professional development experiences of Kentucky teachers and, more importantly, on the differences these experiences may make in teacher and student performance. Because professional development is a significant state investment and the primary means for upgrading teacher knowledge and skills, it is crucial to know what effect it is having on student achievement.
On another front, KDE has been collecting massive amounts of assessment data on students and schools throughout Kentucky through the KIRIS and now CATS systems. Much of this data is distributed to school and district officials, some of whom use it to fashion local responses to assessment results. At the state level, however, these data have not been reviewed systematically or combined with other information on the state’s public schools to identify the factors that contribute most to increased student achievement.
In the reform era, Kentucky has moved to judge school performance based on what students know and can do without real knowledge of what teachers know and can do. At this time, we cannot link the performance, training, or certification of teachers to student achievement. These data limitations are a source of continual frustration to decisionmakers who would like to make recommendations based on solid information, not intuition or best guesses.
Ironically, the issue of teacher quality highlights a more general and pressing concern, namely, the extent to which an assessment-and-accountability-based education system requires school communities to have access to a much broader array of performance information and to develop the facility to analyze and respond to these data. Kentucky educators have been grappling with these data analysis needs during the various cycles of KIRIS and CATS assessments, the results of which largely determine school-level accountability index scores. Given this high-stakes accountability system, educators spend more and more time examining KIRIS scores. Likewise, the new annual School Report Card promises to elevate the importance of school-level data.
Increasingly, teachers and some legislators are calling for student accountability to be incorporated under KERA. Teachers, and indirectly principals, now bear the brunt of the accountability system, with the threat of sanctions if index scores do not improve at an acceptable rate. The CATS system is generating increasingly reliable and valid information on student-level performance, but many argue that only when the results lead to meaningful consequences will students take the exams more seriously.
The data needs of state education agencies are also rising. Likewise, the EPSB, Kentucky’s main teacher policy-setting entity, needs to fill information deficits. At present, EPSB cannot link teacher coursework or performance with Praxis scores, certifications, or professional development needs. Moreover, the data needs of the Council for Postsecondary Education (CPE) are growing. The CPE has long kept information about students who enter Kentucky’s postsecondary institutions, but policymakers want a more seamless education data system that links the postsecondary performance of students to elementary and secondary experiences. These data, they believe, can help us develop policies that enable the greatest number of Kentuckians to achieve optimal education attainment.
Hence, at multiple levels throughout the public education system, data needs are becoming more palpable. What Kentucky needs, in essence, is an education data infrastructure capable—in time—of answering policy questions. Without such information, decisionmakers in Kentucky will be forced to continue setting education policies without a firm evidentiary base. Among the policy questions that an adequate data structure could answer:
Where and how much out-of-field teaching is taking place, and how is it affecting student achievement?
How are teacher assignment practices affecting teaching, learning, and student achievement in Kentucky schools?
What is the teacher supply-and-demand situation in Kentucky? Are recruitment and retention policies in some schools and districts more effective than in others?
What are the professional development needs of Kentucky teachers? If those needs are being met more effectively in some places than in others, what policies might strengthen professional development services in needy areas?
To what extent is student achievement, as measured on CATS or other assessments, related to teacher quality?
How are factors associated with teacher preparation and certification—such as the college attended, test scores (e.g., ACT, Praxis, and GPA), student teaching, induction experiences—linked to teaching performance?
It is unlikely that any data infrastructure will answer all these policy questions in the near future, but it is important to work toward development of a system capable of doing so. Without such a system, Kentucky policymakers will be obliged to make decisions based on hunches and inferences.
KDE collects a massive amount of information about many aspects of public schooling in the Commonwealth; however, much of it cannot be easily obtained or analyzed. The current problem with data might best be described as one of fragmentation, as most KDE data collection has arisen in response to new programs or policies, sometimes along division and program lines, but always within the technological limitations of the times.
These internal dynamics have led to an array of problems. First, it is difficult for KDE leadership to see the full scope of indicators of education system health. Second, data gathering activities sometimes overlap, so different offices ask for similar information at different times of the year, imposing needless hardships on school and district personnel. Third, KDE program data tends to be viewed as proprietary, making it unavailable for review and analysis.
While attempts have been made to address some of these problems, they have sometimes led to the creation of even more databases, or to the transfer of education data to state agencies outside of KDE. Moreover, some of the databases within KDE are so closely linked to one another that it would be difficult to fix problems in one database without addressing problems in others at the same time.
A traditional approach to eliminating fragmentation would be to replace the many disparate databases within KDE with a single, centralized, large-scale database. Constructing such a system, however, would be extremely expensive. Other states have recently devoted from $12 to $20 million on such systems. Such a system can lead to an array of problems, including the need to simultaneously find hardware/software solutions to all of the database problems across the institution. A single system could also prove inflexible if programmers make decisions that inhibit the ability to accommodate change. Moreover, given rapid changes in technology, the system’s hardware/software infrastructure may well be obsolete or insufficient to meet rising data demands by the time it is constructed. Thus, the state could be saddled with a second-rate data infrastructure that it must live with for many years until additional resources for yet another improved system are allocated.
As an alternative to a single system, members of KDE’s Division of Integration Services proposed that KDE build an enterprise database system accessible through a web-oriented “portal,” the sort that many private sector businesses and corporations are now creating, that can gather and sort data from the existing fragmented systems within the Department. Such a portal system would in effect create a unified virtual database that mimics a large-scale data system without the expense, decisionmaking dilemmas, and general constraints the latter system would entail.
An enterprise database and web portal system would be housed on a reasonably powerful PC server located within KDE’s Division of Integration Services. The heart of the system would be a set of software tools designed to (1) extract relevant information via the Internet from various databases located on other servers or PCs inside and outside of KDE, and (2) use software that processes the data it gathers through so-called transformation tables, that reformat the data so the system can work with all data elements, and place data into specified locations within a relational database so that the information can be accessed. This step would allow the data to be “warehoused,” accessed, and mined within appropriate security restrictions. Another set of software tools would allow individuals with an Internet connection to access portions of the data through a web-based user interface.
An enterprise database and portal approach to solving current data fragmentation problems has several advantages:
Allows step-by-step database reconstruction, effectively delinking extant databases so important data remains available while construction is underway.
Does not disturb underlying data systems, providing stability within KDE and among school and district staff who submit data to KDE.
Provides better access to a broader range of data that could enhance local involvement in schools, a primary but unrealized goal of KERA, and enable more informed policymaking at the local, district, and state levels of governance.
Reduces the data gathering burden on schools and districts, eliminating redundant data requests and enabling electronic submissions of data.
Helps resolve concurrency problems that create significant lag times between data submission and reporting, permitting more timely data gathering and verification.
Allows for eventual creation of a more traditional consolidated database. Whatever the technological terrain of the future, the enterprise database and portal system will likely be more adaptable than a traditional system based on current technology.
Provides a mechanism for interagency data sharing and cooperation among relevant agencies and institutions.
Reduces data system costs. Since access by users is based on web technology, rather than specialized software, ongoing costs would be more modest. In addition, the enterprise database and portal system can actually be constructed through discrete database renovation projects, further streamlining costs. Moreover, the new system could reduce data gathering burdens on school, district, and KDE staff.
Aligns with other plans to move schools toward a uniform student data management software program.
The enterprise database and portal infrastructure is the most practical, efficient, and promising way to enhance the Commonwealth’s education data capacity. Information about teachers’ certifications, coursework, and professional development backgrounds could help improve teacher and student performance across the state. Moreover, a well-designed and well-constructed system could become a national model, providing a wealth of data to policymakers, teachers, administrators, researchers, parents, and citizens. While it may take years to unfold, the benefits to the Commonwealth’s public school system should be innumerable.