...
| Info | |
|---|---|
|
...
...
| |
The Assessment Engine allows LearningBuilder to administer on-line, low-stakes assessments that are typically used for Practice Exams and other supporting evaluations. |
Assumptions
- The Assessment Engine is not designed to manage highly secure tests. We want to make it annoying to hack the assessments, but not dedicate significant resources to preventing cheating.
...
...
...
 Yes | ||
| 2 | Create a Section | As an Administrator, I create a Section of questions that groups a set of questions around common content. |
...
| Yes | ||
| 3 | Create a Question | As an Administrator, I create a Question so that I can measure a specific component of knowledge. |
...
| Yes | ||
| 4 | Re-Order Questions | As an Administrator, I order questions so that there is a sequential experience of questions. |
...
| Yes | ||
| 5 | Create a Response | As an Administrator, I create a Response so that a person can answer a question. |
...
| Yes |
...
...
| Yes | ||
| 7 | Specify a Competency Model | As an Administrator, I assign a Competency Model to the Assessment to govern the scope of Competencies available to assign to Questions. |
...
| Yes | ||
| 8 | Define Result | As an Administrator, I manage a Results page so that the results can be tailored to the kind of assessment being taken. |
...
| Yes | ||
| 9 | Publish an Assessment | As an Administrator, I publish an Assessment so that it is available to be completed. |
...
| Yes | ||
| 10 | Un-publish an Assessment | As an Administrator, I un-publish an Assessment so that it is not available to be completed while changes are made. |
...
| Yes | ||
| 11 | Automated Scoring | As an Administrator, by indicating the "correct" answer for each question (or by assigning each answer a value and specifying a Cut Score for the Assessment as a whole), I can create an Assessment that is immediately and automatically scored so that the user receives immediate realtime feedback about their performance. |
...
| Yes | ||
| 12 | Manual Scoring | As an Administrator, I indicate that certain questions (such as essays or other complex interaction types) require human evaluation before a score can be assigned. |
...
 Not yet | Designed, but not implemented | |
| 13 | Manage Forms | As an Administrator, I create multiple Forms within a single Assessment so that I can administer the Assessment multiple times without duplicating the Assessment Experience. |
...
| Not yet | Designed, but not implemented | |
| 14 | Specify that the Assessment has a Time Limit | As an Administrator, I specify the amount of time in minutes a person is allowed to complete the Assessment so that the Assessment is not open indefinitely. |
...
 Partially | Current supports per-question time limit only | |
| 15 | Track assessment-level timing metrics | System records the amount of time |
...
...
...
| Yes | Tracked, but not displayed in the UI anywhere | |
| 16 | Specify that the Assessment will use Confidence Ratings | As an Administrator, I specify that questions will include a Confidence Rating to understand how confident the Test Taker is in the select answer during an Assessment |
...
| No | Can be modeled using custom Questions, but no way to present those questions only a percentage of the time | |
| 17 | Specify the Confidence Rating Scale | As an Administrator, I specify the Rating Scale used for the Assessment so that my ratings can use language and specificity I am comfortable with. |
...
| Partially | Scales are managed per-question. There is no facility for centrally defining a scale and then reusing it. | |
| 18 | Specify that the Assessment will use Relevance Ratings | As an Administrator, I specify that questions will include a Relevance Rating to understand how relevant the Test Taker believes the content is to his or her practice. |
...
| No | Can be modeled using custom Questions, but no way to present those questions only a percentage of the time | |
| 19 | Specify the Relevance Rating Scale | As an Administrator, I specify the Rating Scale used for the Assessment so that my ratings can use language and specificity I am comfortable with. |
...
| Partially | Same as Confidence rating | |
| 20 | Answer Confidence Rating | As a Test Taker, when I answer a Question and Confidence Ratings are turned on, I will answer my Confidence Rating so that I can share my level of confidence in my answer with the Board |
...
| No | ||
| 21 | Answer Relevance Rating | As a Test Taker, when I answer a Question and Relevance Ratings are turned on, I will answer my Relevance Rating so that I can share my belief in the relevance of the Question to my practice. |
...
| No | Same as Confidence. | |
| 22 | View Time to Complete | As a Test Taker or an Administrator, I can see how long it took to complete a sitting of an Assessment |
...
| Partially | Timing data are tracked, but not displayed. (Can be exposed using custom reports) | |
| 23 | View Time on each Question | As an Administrator, I can see how long each test taker took to complete each Question. |
...
| Partially | Timing data are tracked, but not displayed. (Can be exposed using custom reports) | |
| 24 | Record Item Response Strings | As the System, I record the responses to each Question in the QTI standard for the Item type so that the results can be shared with any system using the QTI standard |
...
| Not yet | A QTI export could be constructed using custom reports or the Integration Hub, but no native QTI representation exists. | |
| 25 | Record Post-Test Surveys | As a Test Taker, I can complete a Post-Test Survey so that I can provide feedback to the Board regarding the Assessment Experience. |
...
| No | This can be done using using Workflow Questions instead of |
...
User interaction and design
Questions
Below is a list of questions to be addressed as a result of this requirements document:
...