6 Part 6: V-Modell Reference Activities

6.3 Activities

6.3.5 Evaluation

6.3.5.8 Evaluating Usability

Purpose

Within the framework of usability testing it shall be determined whether an application will be fit for use. For example it shall be ensured that all required information is displayed, that the order of the fields is correct, that the dialog sequences are clear and that all terms are precisely formulated and comprehensible for the user.

Testing will start with the performance of the usability tests. In these tests application cases, each consisting of the description of an application situation and a work task, shall be displayed to the user on a separate screen, such as a notebook, and read out to him in a loud voice. Then the user will deal with the respective task, instructed and supported by a person responsible for human engineering matters and sitting next to the test subject. Alternatively, a notebook with tasks may be specified that will have to be worked off in the course of the test. In each case problems, open issues, wishes, impressions and errors shall be directly addressed and recorded.

It has proved to be worthwhile to seek expert opinions and expert reviews concerning the dialog concept prior to conducting the user test and to take them into account in the validation.

During the test for example the method of thinking out loud in which everything the user thinks and feels while working on the task will be expressed in a loud voice.

During testing it has proved to be successful to start working on individual interface components, such as input fields, lists and menus, and then to check whether these elements are appropriate for use. This will apply also to overarching aspects, such as the nesting of windows, the arrangement of information and the distribution of functions among buttons and menus.

In the follow-up of the tests a small group of people (»Ergonomics Manager and developers) should once more review all records with regard to critical scenarios. Problems, open issues, but also design decisions that proved to be good, should be documented.

In the evaluation report the results of the evaluation cases with regard to usability shall be laid down. The documented results may be used again in the further course of interface implementation as a checklist for design decisions that still will have to be made or that have already been made.

If it turns out that errors occurred during the test, those errors shall be evaluated and prioritized before changes in the development process will be implemented.

6.3.5.8.1 Verifying Usability

The evaluation object has to be verified based on its evaluation specification. In this context evaluation objects may be either arbitrary system elements or even (partial) deliveries. During the verification all evaluation cases defined in the evaluation specification have to be realized. For the system elements system, »Segment, SW unit/HW unit, SW component/HW component and SW process module/HW process module the corresponding evaluation procedure has to be completed. During the evaluation the associated evaluation report has to be prepared. In this record the result achieved for each evaluation case has to be documented such that it can be replicated. The result has to be compared with the expected result. Deviations from the expected result have to be marked as errors. For each evaluation case a summary evaluation has to be provided.

If the test was passed, a state change from "submitted" to "accepted" has to be arranged, otherwise a change from "submitted" back to "in preparation".

6.3.5.8.2 Validating Usability

During the validation the recipient of the evaluation object, i. e. the acquirer or the »System Integrator, shall check whether the evaluation object meets his expectations and has the properties required for the planned use. Evaluation objects may be in this case either (partial) deliveries or any system element.

During the validation the »Inspector of the evaluation object may conduct tests in any sequence and depth, and the evaluation object should always provide an acceptable evaluation result. Depending on the state of manufacture of the evaluation object, a validation may be conducted

• in the form of a simulation based on intermediate results;
• in the form of an evaluation of the planned properties of a system element based on a prototype;
• as an evaluation step within the framework of an execution decision, for example at the end of a development step during incremental system development;
• in the form of an evaluation of the completed evaluation object.

The results of the validation shall be written down in the evaluation report.

If the result of the validation is negative, the acceptance of the evaluation object will be reduced. One reason for this may be that between project start and the time of the evaluation the expectations of the acquirer with regard to the functionality of the product have changed. This may be discovered early if validations are performed during the whole system development process.