Exam Scheduling / Self-Proctored Exams

Summary

LearningBuilder can be used for exam scheduling using the Activity Offerings feature. This feature handles the coordination and orchestration between four parties:

Credentialing Progams that choose when to offer exams
Exam Sites that opt-in to specific exam dates and choose where to offer the exam, including the number of candidates they can support
Candidates that choose from (or are assigned to) available exam offerings
Proctors that manage the exam administration

Terminology (for the "Exam Scheduling" use case)

TODO: Org Staff role

Term	Meaning
Exam Administrator	The Exam Administrator is the entity that produces the exam, certifies the results, etc. For instance, this might be an organization offering a credential that is earned by passing the exam. These entities should be represented in LearningBuilder as Providers.
Exam Activity	Activity Definitions represent specific Exam Administrations that will be offered. For instance, if the "Certified Physical Therapist" exam is offered twice a year, then there might be one Activity Definition that represents "CPT Exam - Spring 2019" and another that represents "CPT Exam - Fall 2019". The Exam Activity is "owned by" the Exam Administrator.
Exam Site	Each entity that offers exam sessions should be represented in LearningBuilder as a Provider. These exam sites might be colleges, testing centers, etc.
Activity Offering	Activity Offerings link Exam Activities and Exam Sites together. Each Offering: Is owned by a specific Exam Site Occurs at a specific date/time/location Has a specific "Capacity" representing the number of people that can be accommodated
Proctor	LearningBuilder supports two proctor models: one where the Proctors are direct staff of the Exam Administrator entity, and another where Proctors are third party contractors that can provide proctoring services to multiple Exam Administrators.

Prerequisites

LearningBuilder 9.10 or later

Capabilities

Exam Sites can "opt-in" to specific exam dates, or administrators can manage this process for them
Candidates can either select a specific "offering", or they can express preferences and have the system automatically assign them to an offering based on availability and custom business rules
Candidates can request accommodations, and approved accommodations can impact the seat assignment strategy (e.g., someone requiring a translator may "take up more space" than someone that doesn't)
If a given exam offering doesn't have enough assigned candidates to be feasible, administrators can easily cancel it and automatically re-allocate the assigned candidates to other exam offerings
Proctors can print attendance reports and track data or provide feedback directly within LearningBuilder

Known Limitations

"Out of the box" implementations do not support rules-based distribution of candidates to available Offerings. If custom assignment logic is required, those rules will be implemented in a custom SQL procedure created by Heuristics technical staff.

Creating the Exam Activities

(For more details, see the Activity Offerings page)

Create a Role representing the Exam Administrator. This should have access to the Admin → Activities or Provider → Activities areas of the site.
1. Assign Grant Role and Edit Role workflows
2. Grant the "Manage Offerings" permission to this Role
3. Create at least one Member with this role, for testing purposes.
Create a Role representing the Exam Sites. This should have access to the Provider → Activities area of the site.
1. Assign Grant Role and Edit Role workflows
2. Grant the "Manage Offerings" permission to this Role
3. Create at least one Member with this role, for testing purposes.
Create an Activity Type to represent the Exam.
1. Leave the Offerings feature disabled for now; the Activity Type must be created before the Create Activity Workflow can be stubbed in.
Create Workflow placeholders:
1. A Create Activity Workflow that will be used to create the Exam Administration Activity
2. A Create Offering Workflow that will be used to create the individual exam offerings
3. (You might also have to create a dummy Complete Activity Workflow for the Activity Type to be considered "Active", but it has no bearing on this use case)
Update the Activity Type that represents the Exam.
1. Enable the Offerings feature for this Activity Type
2. In the "Allow Offerings to be created by" field, select both the Exam Administrator and Exam Site Roles.
3. In the "Allow Offerings to be owned by" field, indicate that 3rd party ownership is allowed.
Configure the Create Activity workflow.
1. Step 1 must be owned by the Exam Administrator Role
2. There are no specific fields that need collected.
3. If you want the Exam Site members to create their own Offerings, you might want to configure a Send Email to Role behavior that notifies them of the newly available Exam Activity. (See below under "Creating Offerings")
Configure the Create Offering workflow.
1. Step 1 must be owned by the Exam Site Role
2. Be sure to include the "Capacity" field, which must be collected.
3. If you'll be doing online scheduling, you should probably also collect Starts On, which is the date/time of the Offering.
Log in as the Exam Administrator and create separate Activity Definitions for each Exam Administration. (For example, if the Certified Financial Planner exam is held twice a year, then there might be one Activity called "CFP Exam - Spring 2019" and another called "CFP Exam - Fall 2019")

Creating Offerings

There are multiple ways to create new Offerings, although they cannot all be configured simultaneously because they require the 1st step of the Create Offering Workflow to be owned by different Roles.

Use Case	Use Case	Configuration Notes
Exam Sites manually create Offerings	Exam Site logs in and goes to Provider → Offerings Exam Site clicks an "Add Offering" button Exam Site sees a list of available Exam Activities and selects one Exam Site fills out the Create Offering Workflow If the user is allowed to create Offerings, but doesn't have permission to act on the first step of the Create Offering Workflow, then the Offering will be created but the Workflow Popup will not appear.	The Provider → Offerings page is an instance of the Activity Offering Search framework that is configured to show Offerings owned by the current user. The "Add Offering" button is a custom button controlled via the search config (managed in Sys Admin → Configuration → Workflow Object Search) To disallow Exam Sites to create their own Offerings, omit the custom button from the configuration
Exam Sites are notified via email of new Exam Activities & follow a link to "opt-in"	Exam Administrator creates a new Exam Activity All registered Exam Site members receive an email with a link Exam Site clicks the link and is taken to their Provider → Offerings page, and the "Add Offering" popup automatically appears (From this point on, same as if they had clicked the button manually) If the user is allowed to create Offerings, but doesn't have permission to act on the first step of the Create Offering Workflow, then the Offering will be created but the Workflow Popup will not appear.	Email notification is handled via a Send Group Email Behavior that sends an email to the Exam Site Role The email template uses a special URL parameter to trigger the popup
Exam Admin creates Offerings; Exam Sites provide all the details	Exam Administrator creates a new Exam Activity Exam Administrator accesses the Manage Offerings page for that Activity and clicks the "Add Offering" button Exam Administrator selects an "owner" from a list of Exam Sites that are eligible to "own" the Offering The popup closes, and the Offering is displayed in the list. The Create Offering Workflow Popup does not automatically open because the Exam Administrator does not have rights to the 1st step
*Exam Admin completes Create Offering* Workflow on behalf of the Exam Site**	(Offering is create same as above, in which the Exam Admin creates it but does not launch the Workflow Popup) After creating the Offering, the Exam Admin goes to Admin → Members → (Exam Site) → Offerings to view a list of Offerings owned by a specific Exam Site Exam Admin clicks the Workflow Button for the newly added Offering and uses their "Act On Workflows For Others" permission to complete the Workflow	Requires that the Exam Administrator Role have access to Admin → Members and have the "Act On Workflows For Others" permission in the Admin area.
Exam Administrator creates Offerings directly	Exam Administrator creates a new Exam Activity Exam Administrator accesses the Manage Offerings page for that Activity and clicks the "Add Offering" button Exam Administrator selects an "owner" from a list of Exam Sites that are eligible to "own" the Offering	Requires that the first step of the Create Offering Workflow is owned by the Exam Administrator Role, which *prevents Exam Sites* from initiating the creation process** The Exam Site should not be listed in the "Allow Offerings to be created by" setting on the Activity Type.

Adding Seat Assignments to a Workflow

An "Exam Offering" indicates that a specific Site will be holding an exam at a specific date/time/location, and the location has a specific number of people that it can accommodate. Candidates are assigned to those offerings through the Seat Assignment attribute.

There are multiple ways in which the Seat Assignment attributes can be managed:

Use Case

Configuration Notes

Candidate does scheduling as part of the Complete Learning Plan Workflow

Candidates add Activities to their application as normal. The Seat Assignment is part of the application data.

Add an Activity Filter attribute to the built-in Configure Learning Plan Workflow. This will be used to control which types of Offerings can be selected.
Trigger the configuration workflow for the Learning Plan Definition(s) that will participate in the scheduling process. Configure the Activity Filter attribute to choose the Exam Administration Activity Type.
You can optionally choose to allow the Activity Start/End dates to specify the "Scheduling Window". If this option is selected, then Practitioners will only be able to choose from Offerings for Activities when the current date falls between the Activity Start and Activity End range.

Candidate does scheduling as part of a Complete Activity Workflow

If Activities represent Exams, then the Seat Assignment would be added to an Activity Instance and the scheduling would occur as part of that workflow.

Same as above, except that the Activity Filter is added to a Create Activity Workflow, rather than the Configure Learning Plan.

Seat Assignment Approval Process

There are different models through which the Seat Assignments are requested and approved.

Use Case

Configuration Notes

Candidate chooses a specific Offering which is automatically confirmed

Candidates are shown a list of Exam Sites offering the exam and the dates/times/locations associated with those Offerings.

Candidates choose an Offering from the list, and the seat assignment is automatically marked as "confirmed". No back-end review is necessary.

Add a Seat Assignment attribute to a Complete Learning Plan or Complete Activity Workflow, and select the relevant Activity Filter (see above)
Configure the attribute so that the seat assignment has a default status of "confirmed"

Candidate chooses a specific Offering. Administrators manually approve the request.

Candidates are shown a list of Exam Sites offering the exam and the dates/times/locations associated with those Offerings.

Candidates choose an Offering from the list, and the seat assignment is automatically marked as "pending".

Exam Administrator reviews the data and either manually approves the seat assignments one-by-one (by accessing the Workflow Popup with a Role capable of modifying the Seat Assignment status) or by using the bulk approval features on the Manage Offerings page.

Same as above, plus:
Configure the attribute so that the seat assignment has a default status of "pending", and that the desired Role is allowed to modify the assignment status
Ensure the attribute is displayed in input mode on a Workflow Step that the desired Role can access, so that the status can be manually modified. This may require creating an approval queue.

Candidates indicate their preferences and are assigned specific seats based on custom business rules

Candidates do not choose a specific Offering. Instead, they indicate a set of preferences using custom attributes.

After all of the Candidates have registered, the requests are examined and sorted by a custom algorithm that creates the Seat Assignment data and assigns all Candidates to an Offering in a "pending" status.

Exam Administrator reviews the assignments. If needed, they make changes to the data and re-run the allocation algorithm.

Once satisfied, the Exam Administrator uses the bulk confirmation features on the Manage Offerings page to mark all Seat Assignments as "confirmed".

Use custom Workflow Attributes to collect preference data
Create the Seat Assignment attributes, but hide them from the Candidate. (They should remain visible to Exam Admins to allow for manual overrides)
Create a custom stored procedure that implements the business rules and associate it with the Activity Type

Managing Offerings

(For more information on these pages, see Offerings → "Managing Offerings")

Page	Purpose
Admin → Activity List → Manage Offerings	Administrative review of an Exam Activity Administrator providing support to an Exam Admin or Exam Site
Admin → Members → Member Offerings	Administrator providing support to an Exam Proctor
Provider → Activity List → Manage Offerings	Exam Administrators manage an Exam Activity; viewing reports, managing seat assignments, etc.
Provider → Activity List → All Offerings	Exam Sites manage their exam Offerings
Provider → Activity List → My Offerings	Proctors that are direct staff of a specific Provider can see a list of exams they are assigned to Proctors that are direct staff of multiple Providers can see a list of exams they are assigned to, segmented by Provider
My Account → My Offerings	Proctors that are 3rd party contractors and NOT staff of a specific Provider can see a list of all exams they are assigned to, across all Providers

Configuring Special Accommodations

TODO

LearningBuilder Documentation