We introduce a new usability walkthrough method called Collaborative Critique (CC) that is inspired by the human-computer collaboration paradigm of system-user interaction. This method applies a "collaborative lens" to assessing the system's behavior and its impact on the user's efforts in the context of the task being performed.
Collaborative Critique is a walkthrough method that involves evaluators going through a step-by-step scenario of using the system to perform a task while answering a set of questions. See the description of the method and examples here.
Usability inspection methods, such as Heuristic Evaluation (HE), Heuristic Walkthrough (HW) and Cognitive Walkthrough (CW), remain a key technique, although researchers and practitioners have acknowledged their deficiencies. These deficiencies include limiting the usability analysis according to a set of heuristics or specific user difficulties; the limited consideration of the context of the system-user interaction; and little analysis of problem causes, leading to difficulty in generating appropriate fixes. The CC method aims to mitigate some of these problems by explicitly invoking the context of the interaction in the evaluation, including the task context and the user's experience and role. It considers the total effect of the interaction on the user's cognitive and physical efforts and evaluates specific components of the system's behavior.
We have evaluated the method in a laboratory setting by comparing the predictions of the Collaborative Critique to the results of user testing. The results of this evaluation are presented in this CHI paper.