This year's FCAT scoring, with late resultschallenged by many superintendents, did little to reassure Floridians that the state's school accountability system was achieving the desired results. Hoping to instill more confidence in the public -- not to mention teachers, who soon could find these test results impacting their pay rates -- Florida education commissioner Eric J. Smith convened several advisory committees to seek out ways to improve the system. The Leadership Policy Advisory Committee for Assessment and Accountability met Wednesday in Jacksonville, and plans to meet again in a few months. Smith spoke with reporter Jeff Solochek about the first session and what he hopes to accomplish.

Why did you even have this meeting?

The superintendents have raised issues about the accuracy of the FCAT. I think we got those resolved. But as follow-up, there certainly are other questions that they raised that persist. For example, we do see in some of our score reporting fluctuations in results from year to year. We'd like to see if we can't reduce some of those fluctuations. The bottom quartile numbers, for example, were accurate as the auditors found, but is there a technical way that we can reduce that fluctuation to have more consistency in scores?

... Is there strong equating from year to year from elementary to middle to high school? Because the results you see is that high school kids are not doing as well, so does that mean the elementary should be strengthened? Or is high school not equated to the elementary accurately? And so forth. So some of these questions have been raised in the process of working with the superintendents. I think they're all very valid and appropriate questions to be raised. Right now it's the perfect time to raise them and really analyze them to see if there's a way to improve our testing and accouintability program. We're preparing to go into what we call FCAT 2.0 this spring, so there are some near term issues, recommendatios adn so forth that we should take time to explore as they relate to this year's test. There are also some long-term recommendations or things we should explore as it pertains to a year from this spring.

What are some of the short-term things that you are looking at? Did they raise very specific questions?

Oh yeah. It was an extremely productive meeting. Very productive and very positive. Some of the questions they had, had to deal with the bottom quartile, where you see the fluctuations from year to year. One question would be, would it increase the accuracy if we maybe did an averaging of the bottom quartile scores over a three year period. So that we could minimize some of that fluctuation and level it out to be able to show if a school over time is making progress with its bottom quartile or if it's not doing as well with its bottom quartile kids. But by doing it over time, get a more accurate snapshot of the work being done by the school. That was one example of a question that would be a near term one. If it looks like it might give us a better result or a result that educators have more confidence in, could that adjustment be made for this year's assessment and accountability rollout?

Did you get a sense that people are still mistrustful of the results?

I think the meeting was extraordinary. It was not a meeting where we were looking backward. We were looking forward. To a person in that meeting, in my perspective, everyone was in complete agreement about the importance of accountability. No one is backing away from a strong accountability program and a strong testing program. I think everybody in the room ... were interested in trying to find ways to make it even better. So it was a very positive meeting. It was a very constructive meeting. We're going to get back together after we've had the opportunity to do more research to try to bring some data and some analysis to some issues that were raised. I'm very hopeful that the conversation is going to lead to an even stronger position for our state.

Did you talk about how the new FCAT and the issues you were talking about will play into the teacher evaluation system?

We didn't get into that specifically as our topic. But we did all realize that the strength of the FCAT and the confidence in its results are going to be critical if we end up using this for purposes of evaluation and compensation decisions. So getting it right is very very important. And that's where I think everybody is trying to head, to see if we can't strengthen some of the procedures and outcomes we get. Not making anything easier. Not backing away from anything. But strengthening the accountability and testing program to actually give it more credibility and more confidence in it.

How do you see all of this affecting kids in the classroom?

This work at this meeting will have a positive impact on our students. We're looking to not just have the best of class in terms of our test, but to make it even stronger than that. ...

Will they see it in the test? Or in their instruction? Or just in the results?

At this point in time, they will see it in the way we do some of our calculations and in our test construction. There was some discussion that we currently have teachers that serve on committees that do a lot of the test development work. There were suggestions that maybe we need to identify a broader group of stakeholders to do that work. ... There will be some visibility of the meeting and what might come of that. But I think ultimately it will lead to greater confidence.

About the blog

Gradebook features education articles and insights on schools in Florida, focusing on Tampa Bay area schools. What's the latest from the Florida Department of Education? How is the FCAT being used to compare Florida schools? What's going on in Tampa Bay schools? Get an insider's view from the Times education reporting team.