User Experience Trust Metrics

Introduction

UX Metrics should enable measurement of the evolving baseline for participation in the Identity Ecosystem.

The requirements for this page are based on a TFTM presentation of October 2014 and are intended as input to the TFTM process. The following questions are taken from that presentation.

What is the baseline?

Improving the security, privacy, usability, and interoperability of everyday online transactions

What benefits could the everyday consumer see if this baseline was established?

e.g., reduced account compromise through increased use of multifactor authentication; greater user control through notice, consent requirements; etc.

Content

The following are the points where user experience can be measured to determine if the base line requirements are met. Note that the first metrics are for overall usability moving into the trust measurements that will indicate compliance of a particular implementation with the terms of the IDESG or of the Framework.

Measurements

1. Can the user accomplish the task set out to accomplish? (A goal might be 90%, a minimum acceptable might be 60%)
2. What is the System Usability Scale (John Brooke's SUS)
3. It the Trustmark discoverable and self-describing. (90%, 70%)
4. Does the user feel safer as a result of the appearance of the Trustmark. (99%, 80% of those answering in the affirmative above.)
5. Does the user feel that the site is safe overall? (the metric is a comparisons of the positives to the negatives.)
6. Does the user understand the necessity for a strong identity for their providers?
7. Does the user know whether the identity of the provider is strongly bound to a real-world entity?
8. Collected verbatim responses used for site improvement.

The System Usability Scale

The SUS is a 10 item questionnaire with 5 response options.

1. I think that I would like to use this system frequently.
2. I found the system unnecessarily complex.
3. I thought the system was easy to use.
4. I think that I would need the support of a technical person to be able to use this system.
5. I found the various functions in this system were well integrated.
6. I thought there was too much inconsistency in this system.
7. I would imagine that most people would learn to use this system very quickly.
8. I found the system very cumbersome to use.
9. I felt very confident using the system.
10. I needed to learn a lot of things before I could get going with this system.