| # | Title | User Story | Implemented? | Notes |
|---|
| 1 | Create an Assessment | As an Administrator, I create an Assessment so that I can evaluate a person's knowledge. |  Yes |
|
| 2 | Create a Section | As an Administrator, I create a Section of questions that groups a set of questions around common content. | Yes |
|
| 3 | Create a Question | As an Administrator, I create a Question so that I can measure a specific component of knowledge. | Yes |
|
| 4 | Re-Order Questions | As an Administrator, I order questions so that there is a sequential experience of questions. | Yes |
|
| 5 | Create a Response | As an Administrator, I create a Response so that a person can answer a question. | Yes |
|
| 6 | Specify a Cut score | As an Administrator, I create a cut score to establish the score necessary for a passing result. | Yes |
|
| 7 | Specify a Competency Model | As an Administrator, I assign a Competency Model to the Assessment to govern the scope of Competencies available to assign to Questions. | Yes |
|
| 8 | Define Result | As an Administrator, I manage a Results page so that the results can be tailored to the kind of assessment being taken. | Yes |
|
| 9 | Publish an Assessment | As an Administrator, I publish an Assessment so that it is available to be completed. | Yes |
|
| 10 | Un-publish an Assessment | As an Administrator, I un-publish an Assessment so that it is not available to be completed while changes are made. | Yes |
|
| 11 | Automated Scoring | As an Administrator, by indicating the "correct" answer for each question (or by assigning each answer a value and specifying a Cut Score for the Assessment as a whole), I can create an Assessment that is immediately and automatically scored so that the user receives immediate realtime feedback about their performance. | Yes |
|
| 12 | Manual Scoring | As an Administrator, I indicate that certain questions (such as essays or other complex interaction types) require human evaluation before a score can be assigned. |  Not yet | Designed, but not implemented |
| 13 | Manage Forms | As an Administrator, I create multiple Forms within a single Assessment so that I can administer the Assessment multiple times without duplicating the Assessment Experience. | Not yet | Designed, but not implemented |
| 14 | Specify that the Assessment has a Time Limit | As an Administrator, I specify the amount of time in minutes a person is allowed to complete the Assessment so that the Assessment is not open indefinitely. |  Partially | Current supports per-question time limit only |
| 15 | Track assessment-level timing metrics | System records the amount of time it takes to answer each Question. | Yes | Tracked, but not displayed in the UI anywhere |
| 16 | Specify that the Assessment will use Confidence Ratings | As an Administrator, I specify that questions will include a Confidence Rating to understand how confident the Test Taker is in the select answer during an Assessment | No | Can be modeled using custom Questions, but no way to present those questions only a percentage of the time |
| 17 | Specify the Confidence Rating Scale | As an Administrator, I specify the Rating Scale used for the Assessment so that my ratings can use language and specificity I am comfortable with. | Partially | Scales are managed per-question. There is no facility for centrally defining a scale and then reusing it. |
| 18 | Specify that the Assessment will use Relevance Ratings | As an Administrator, I specify that questions will include a Relevance Rating to understand how relevant the Test Taker believes the content is to his or her practice. | No | Can be modeled using custom Questions, but no way to present those questions only a percentage of the time |
| 19 | Specify the Relevance Rating Scale | As an Administrator, I specify the Rating Scale used for the Assessment so that my ratings can use language and specificity I am comfortable with. | Partially | Same as Confidence rating |
| 20 | Answer Confidence Rating | As a Test Taker, when I answer a Question and Confidence Ratings are turned on, I will answer my Confidence Rating so that I can share my level of confidence in my answer with the Board | No |
|
| 21 | Answer Relevance Rating | As a Test Taker, when I answer a Question and Relevance Ratings are turned on, I will answer my Relevance Rating so that I can share my belief in the relevance of the Question to my practice. | No | Same as Confidence. |
| 22 | View Time to Complete | As a Test Taker or an Administrator, I can see how long it took to complete a sitting of an Assessment | Partially | Timing data are tracked, but not displayed. (Can be exposed using custom reports) |
| 23 | View Time on each Question | As an Administrator, I can see how long each test taker took to complete each Question. | Partially | Timing data are tracked, but not displayed. (Can be exposed using custom reports) |
| 24 | Record Item Response Strings | As the System, I record the responses to each Question in the QTI standard for the Item type so that the results can be shared with any system using the QTI standard | Not yet | A QTI export could be constructed using custom reports or the Integration Hub, but no native QTI representation exists. |
| 25 | Record Post-Test Surveys | As a Test Taker, I can complete a Post-Test Survey so that I can provide feedback to the Board regarding the Assessment Experience. | No | This can be done using using Workflow Questions instead of a 1st class feature. |