Assessment Engine

Create an Assessment

As an Administrator, I create an Assessment so that I can evaluate a person's knowledge.

 Yes

Create a Section

As an Administrator, I create a Section of questions that groups a set of questions around common content.

 Yes

Create a Question

As an Administrator, I create a Question so that I can measure a specific component of knowledge.

 Yes

Re-Order Questions

As an Administrator, I order questions so that there is a sequential experience of questions.

 Yes

Create a Response

As an Administrator, I create a Response so that a person can answer a question.

 Yes

Specify a Cut score

As an Administrator, I create a cut score to establish the score necessary for a passing result.

 Yes

Specify a Competency Model

As an Administrator, I assign a Competency Model to the Assessment to govern the scope of Competencies available to assign to Questions.

 Yes

Define Result

As an Administrator, I manage a Results page so that the results can be tailored to the kind of assessment being taken.

  Yes

Publish an Assessment

As an Administrator, I publish an Assessment so that it is available to be completed.

 Yes

Un-publish an Assessment

As an Administrator, I un-publish an Assessment so that it is not available to be completed while changes are made.

 Yes

Automated Scoring

As an Administrator, by indicating the "correct" answer for each question (or by assigning each answer a value and specifying a Cut Score for the Assessment as a whole), I can create an Assessment that is immediately and automatically scored so that the user receives immediate realtime feedback about their performance.

 Yes

Manual Scoring

As an Administrator, I indicate that certain questions (such as essays or other complex interaction types) require human evaluation before a score can be assigned. 

 Not yet

Designed, but not implemented

Manage Forms

As an Administrator, I create multiple Forms within a single Assessment so that I can administer the Assessment multiple times without duplicating the Assessment Experience.

 Yes

Specify that the Assessment has a Time Limit

As an Administrator, I specify the amount of time in minutes a person is allowed to complete the Assessment so that the Assessment is not open indefinitely.

 Partially

Current supports per-question time limit only

Track assessment-level timing metrics

System records the amount of time it takes to answer each Question.

 Yes

Tracked, but not displayed in the UI anywhere

Specify that the Assessment will use Confidence Ratings

As an Administrator, I specify that questions will include a Confidence Rating to understand how confident the Test Taker is in the select answer during an Assessment

 No

Can be modeled using custom Questions, but no way to present those questions only a percentage of the time

Specify the Confidence Rating Scale

As an Administrator, I specify the Rating Scale used for the Assessment so that my ratings can use language and specificity I am comfortable with.

 Partially

Scales are managed per-question. There is no facility for centrally defining a scale and then reusing it.

Specify that the Assessment will use Relevance Ratings

As an Administrator, I specify that questions will include a Relevance Rating to understand how relevant the Test Taker believes the content is to his or her practice.

 No

Can be modeled using custom Questions, but no way to present those questions only a percentage of the time

Specify the Relevance Rating Scale

As an Administrator, I specify the Rating Scale used for the Assessment so that my ratings can use language and specificity I am comfortable with.

 Partially

Same as Confidence Rating

Answer Confidence Rating

As a Test Taker, when I answer a Question and Confidence Ratings are turned on, I will answer my Confidence Rating so that I can share my level of confidence in my answer with the Board

 No

Answer Relevance Rating

As a Test Taker, when I answer a Question and Relevance Ratings are turned on, I will answer my Relevance Rating so that I can share my belief in the relevance of the Question to my practice.

 No

Same as Confidence.

View Time to Complete

As a Test Taker or an Administrator, I can see how long it took to complete a sitting of an Assessment

 Partially

Timing data are tracked, but not displayed. (Can be exposed using custom reports)

View Time on each Question

As an Administrator, I can see how long each test taker took to complete each Question.

 Partially

Timing data are tracked, but not displayed. (Can be exposed using custom reports)

Record Item Response Strings

As the System, I record the responses to each Question in the QTI standard for the Item type so that the results can be shared with any system using the QTI standard

 Not yet

A QTI export could be constructed using custom reports or the Integration Hub, but no native QTI representation exists.

Record Post-Test Surveys

As a Test Taker, I can complete a Post-Test Survey so that I can provide feedback to the Board regarding the Assessment Experience.

 Partially

This can be done using unscored questions or  Workflow Questions instead of a 1st class feature.