How do I conduct a test question analysis?
Track My Progress is a transparent assessment. This means that you can view the test questions each of your students saw and you can view the correct answer, your student's answer and whether your student skipped the test question.
This transparency allows you to conduct a test question analysis when you need more information as to the exact nature of your student's misunderstandings or a more concrete picture of their skills.
The first step in conducting a test question analysis is to drill into your data to see the list of test questions for your student.
- Sign in to your account.
- Select Groups and the subject or domain (e.g. Reading or Geometry).
- From the Groups graph select your student of concern.
- You will see one or more vertical bars representing each test event for the current school year. Select the most recent test event or the test event for which you would like to see the test questions.
Analyzing the test question list
You will now be viewing the list of test questions for the test event you selected. The test question list view provides several features that facilitate your test question analysis.
There are seven columns of data which can be sorted by clicking on the column heading. Click once to sort. Click again to reverse sort. These column headings allow you to sort on the following criteria:
- Question: The question number in order that the student saw the questions during the test.
- Subject: The Common Core subject of the question.
- Domain: The Common Core domain of the question.
- Standard: The Common Core standard of the test question.
- Level: The difficulty level of the test question. This can be very easy, easy, medium, or hard.
- Time: The time required by the student to complete the question.
- Answer: Whether the student answered correctly, incorrectly, or skipped the question.
Sorting by these column headings enables you to bring certain types of questions to the top to answer your questions about the student's test experience. Here are some examples of concerns you may have about your student's test experience and how you can find answers:
- Did my student skip any test questions?
- A skipped test question will show on the list as a yellow down arrow. If your student skipped more than one or two test questions it could be that your student was not engaged with the test and did not put forth an effort that reveals their best understanding and skills. If there are other sources of data that point to the student being unengaged and not motivated you could look to intervene in those areas as opposed to focusing on the specific content the student skipped.
- Did my student answer any questions in less than five seconds?
- Some students will move through the test very quickly and not carefully consider the test questions. Maybe they want to "finish first" or "just want to be done." Helping the student understand the value of the test to your teaching as well as exploring interventions that address engagement and motivation could be next steps.
- What is the overall trend of the difficulty level of the test questions?
- By reviewing the Level column you can see how the computer adaptive test modified the difficulty of the test questions to fit the learning zone of the student. Most students will start with easy or very easy test questions for the first few. But where does the test go from there? Did the test questions remain easy for a while and then drop to very easy? Do you see in the Standard column the grade level indicator drop frequently below the student's current grade? This is letting you know the test had to branch below grade level to find the learning zone of the student. Or maybe the difficulty level of the questions progressed from easy, to medium to hard. By the end of the test was the student consistently seeing hard test questions? This indicates the student was effectively working above grade level.
You may also place your cursor on top of the information in any of these columns to get a more detailed explanation. For example, if you place your cursor on top of the Standard for a test question in the list a window will appear that provides the full Common Core Standard definition.
Viewing a test question
You can view any test question by clicking the test question number in the left column of the list of test questions. This will open a new window displaying the test question as it appeared to the student. At the top of the test question window are several links that will provide more information when selected:
- Improve this item
- Use this link if you would like to provide feedback to us so we can improve Track My Progress. You may have an idea for how to formulate the question to better measure the standard. Or you may have found something distracting or unfair about the question. Please let us know by using this link.
- See correct answer
- Clicking this link will show the correct answer to the question.
- Common Core Standard
- This alphanumeric code is the Common Core standard for the test question. You can place your cursor on top of the code to see a description of the standard.
- Question number out of total number of questions
- "1 out of 25" means the first test question out of 25 total test questions. There is an arrow to the right of the 25 that you can select to move through the test questions without having to return to the test question list to select the next question.
- See student answer
- Clicking this link will display the student's answer to the test question as she provided it during her test. If the student skipped this question it will not change when you click this link.
- Student answer
- The green check mark, red x or yellow triangle indicates whether the student answered correctly, incorrectly, or skipped this question.
- This is the elapsed time the student required to answer this test question.
Analyzing a test question
Viewing the actual test questions provides you with important information that can help you address your concerns about your students. Here are some ways teachers have been using test question analysis with Track My Progress to better guide their students.
- For students well below grade level, how do I know where to enter for their intervention?
- Students with many very easy test questions in their test question list are commonly 1-2 years below grade level. It can be challenging for a teacher to know how far to drop-back in designing and implementing their intervention. Teachers can first identify the student's lowest scoring domain and then analyze those 5-7 test questions. At what level was the student able to answer correctly and at what level was the student answering incorrectly? The entry level for intervention is between these two levels.
- For students well above grade level, how do I know how to better challenge and engage these students?
- Students with many hard test questions in their test question list are above grade level. To find what kinds of concepts may challenge this student review the questions the student answered incorrectly.
- For students struggling in a specific domain, how do I discover their key misunderstandings or confusion?
- You can view each of the test questions for a specific domain and the student's answer for these questions to determine what misunderstanding may be their barrier to mastery.