Rubric

Keep in mind that 186 students have already been assessed using this rubric. Changing it will affect their evaluations.
User testing of the functional prototype
User testing of the functional prototype
Criteria Ratings Pts
Participants
threshold: pts
10 pts Exemplary The participants recruited for the study represented the personas for the project or the participants engaged in designed role play that effectively simulated the personas.
7.5 pts Accomplished
5 pts Acceptable The participants recruited for the study were only partially representative of the personas for the project.
2.5 pts Emerging
0 pts Unacceptable The participants were not representative or described in relation to the personas for the project.
pts
10 pts
--
Research Questions
threshold: pts
20 pts Exemplary Described and motivated three or more research questions. The questions were meaningful, appropriate, and testable for the project.
15 pts Accomplished
10 pts Acceptable Some of the research questions were not explained well or were inappropriate for the project.
5 pts Emerging
0 pts Unacceptable No questions were described, were not appropriate for the project, or were not comprehensible.
pts
20 pts
--
Method
threshold: pts
10 pts Exemplary The method was appropriate for answering the research questions and to gain insight into the project. The method included descriptions of the participants, role play, tasks, evaluation locations, dates, duration, what was observed, and what interview or survey questions were asked of the participants. There was enough detail that the method could be repeated.
7.5 pts Accomplished
5 pts Acceptable Missed opportunities to fully answer the research questions or gain insight (e.g., due to ineffective or incomplete task design). Alternatively, some critical details of the method (e.g., about the participants, tasks, or procedure) were missing, preventing replicability.
2.5 pts Emerging
0 pts Unacceptable No method was reported or was inappropriate for answering the research questions.
pts
10 pts
--
Data Collection
threshold: pts
10 pts Exemplary Data collection was complete, comprehensive, and insightful in relation to the research questions and project goals. The method of data collection and how the data related to the research questions was clearly explained.
7.5 pts Accomplished
5 pts Acceptable Part of the data collection was unclear, or related to only a subset of the research questions.
2.5 pts Emerging
0 pts Unacceptable No data collection was reported.
pts
10 pts
--
Results and Interpretation
threshold: pts
30 pts Exemplary Results were summarized to answer the research questions and drew from participant feedback, interaction data, and observations of use. The data was aggregated, visually summarized where appropriate, and interpreted in relation to the research questions and project goals.
22.5 pts Accomplished
15 pts Acceptable Results were provided for only some of the research questions, were difficult to understand, or were not interpreted in relation to the research questions.
7.5 pts Emerging
0 pts Unacceptable No results were provided.
pts
30 pts
--
Design Ideas
threshold: pts
20 pts Exemplary Three or more non-trivial design ideas were described from the study results. The ideas were plausible and, if implemented, the design ideas would significantly improve the prototype.
15 pts Accomplished
10 pts Acceptable Three design ideas were described, but some were not plausible or would not improve the prototype.
5 pts Emerging
0 pts Unacceptable The design ideas were not described or were not relevant to the prototype.
pts
20 pts
--