Troubleshooting Using Data to Guide Learning
One of my best readers in my class scored much lower than my other students. What should I do next?
The first step is to go through your student's test question by question to see what kinds of mistakes the student made. This article will show you how to view each test question and the student's answer for a specific test. You can use the forward arrow at the top of the first test question to page through each test question. You can ask the data these questions:
- Did the student skip many test questions?
- Did the student answer some questions in a very short amount of time relative to the complexity of the task?
- Did the student make mistakes that really surprise you given what you know of the student's skills?
- Did the student have difficulty with a particular domain or set of standards that brought down her overall score?
Probably the surest method to unraveling the mystery of an unexpected test score is to sit down with the student and ask her to answer each question as you watch. You can ask her to explain why she choose the choice that she did. You can review with her the answer choice she provided during the actual test and ask her to explain her thinking.
In general there are four common causes of unexpectedly low test scores:
- The student was not engaged in the test, did not try her best or wanted to be finished very quickly.
- The test uncovered a specific domain or area of weakness that brought down the overall score.
- The student struggled with the computer interface and made mistakes related to poor computers skills and not related to poor math or reading skills.
- There was a technical problem during the test and the test questions did not display properly or function properly.
All of the students in my class are in the red. I don't see any instructional groups by different color.
This commonly occurs when teaching a class of students that are well below the national average. What is most important in your use of Track My Progress is your ability to see your most needy students in red (your students who are falling behind their peers in the class) and if possible your instructional groups organized by levels of skill and readiness. To address this issue your school administrator needs to modify the cut scores that define the color categories in Track My Progress. You can learn how to modify the cut scores in this article, and this article will explain how to choose optimal cut scores for universal screening.
What do you recommend if I find that one of my students has skipped a lot of the test questions?
The finding from a test where a student skipped a lot of test questions or guessed on many questions (i.e. spent very little time on each question) is that the student was not engaged in the test process. Track My Progress is a computer adaptive test, which means that each time the student skipped a test question or answered incorrectly the next question is easier. If the student skips or guesses incorrectly at multiple test questions the remaining test questions will become very easy relative to the student's grade level. You can also review the actual test questions to confirm that the student is skipping test questions that are at or below her skill level.
What this means is that the student is not engaged in the test process and possibly not engaged in her larger learning process and goals. If this is the case look to interventions designed to address engagement and motivation issues. Help the student develop personal goals related to her learning in school and how assessments help you understand better how to help her reach her goals.
I created goals for my students using the process described in this article. But one of my students finished the school year below the goal I set for him and below where he started the school year. How is this possible?
There are several important concepts to keep in mind when setting goals for students and using data for educational decision making. The first is the concept of using multiple data points in your decision making. Please see this article that provides an overview of the importance of using multiple data points in educational decision making and also provides a real example similar to what you describe in your question.
It is possible that your student had a very poor academic year and did not make expected progress. If this were the case you would have seen this in the two or threesets of Track My Progress scores over the course of the school year. You would have seen 'flat' or declining scores over the course of the year. And this is why Track My Progress is designed to allow for four test windows a year to avoid the over reliance of leveraging a single test score for educational decisions.
If, on the other hand, this student showed a steady progression through the fall, winter, and spring test windows and then had a very poor test in the summer window it likely means that this was a 'bad day' for the student and he was not able to show his best work. Here, you need to look to other sources of data (quiz scores, test scores, homework quality, attendance, attitude, etc.) to confirm that he is progressing on track or maybe discover that he is not and that he should be followed more closely at the start of the next school year.
Are test scores reliable for kindergarten students who have very little computer experience?
This is an important question and one you can answer by carefully watching your students during their Track My Progress test sessions.
- Do you see your students struggling to drag answers to their intended location?
- Do you find you are constantly moving around the computer lab helping students and showing them what to do?
- Are students more interested in exploring the computer-mouse interface and not really working to answer questions and show what they can do?
Some schools use the fall and sometimes winter test events for kindergarten students as purely an introduction to computer test taking. In other words, they do not use the data for educational decision making. Other schools will not test kindergarten students in the first or second test windows of the year but will work on computer skills.
Your strategy in how you use Track My Progress will depend on your specific students and their familiarity with computers. Some educators find it helpful to compare computer-based assessments for young students to other forms of assessment. For example, we have seen kindergarten students early in the school year who are completely overwhelmed by the one-on-one reading screening with an adult they have only just met. They are not familiar with interacting with adults this way and don't have a conception of what the assessment means. We suggest thinking of the computer-based assessment in a similar way. When a student is placed in a new context for an assessment (whether it is computer-based, one-on-one, or pencil and paper) we have to interpret any resulting scores or findings with this in mind.