Overview

The Workflow Import Queue is a scalable, Workflow-centric background processing queue for importing CSV data into LearningBuilder.

As of Release 12.2, this feature is available in two places:

• Admin → Data Import: Allows Administrators to import various types of data into the system.

• Provider → Data Import: Allows Providers to report Attendance, upload course data, etc.

Primary use case versus legacy data upload features

When compared to the legacy upload features, the the primary benefit of the Workflow Import Queue is that it leverages the Workflow Engine to enable new use cases:

• Validation Rules and Required Fields enforce complex and dynamic validation requirements;

• Uploaded data can trigger Workflow Actions;

• New types of data can be imported (e.g. Learning Plan Instances);

• The Workflow Import Queue supports larger files with more predictable performance.

Bulk CSV upload

Bulk data imports can be performed by uploading a CSV file.

One-by-one data entry

As of Release 12.2 there is an in-app data entry form as an alternative to file uploads.

“Data Import” Workflow Steps

The foundation of the Workflow Import Queue is a new type of Workflow Step called a “Data Import Step”.

A traditional Workflow Step, now called a “Data Entry” Step, describes the form that is displayed to a user within LearningBuilder to collect data at a specific part of a multi-part process.

By comparison, Data Import Steps describe the format for a data file that can be uploaded to create or modify data regardless of the step the entity is currently “on”. In other words, Data Import Steps describe an “out-of-band data manipulation” that can be performed via file upload.

For more information on Data Import Steps, see Data Import Steps

“Identifier” vs “value” fields

The Import Queue is capable of both inserting and updating data. In some cases, it needs to know which data elements represent existing values used to find an existing record to update, and which pieces of data hold new values to use when updating / creating a record.

There are three “types” of data elements:

• Value fields

• Identifier fields (“:” prefix)

• Comment fields (“#” prefix)

Value Fields (no prefix)

Value fields contain the new data that is being used to insert or update records in LearningBuilder. When uploading a CSV file, column headers without a special prefix are value columns.

Typically, each Attribute on the target Data Import Step should have a corresponding value column in the uploaded file, or a corresponding input field on the in-app data entry form.

Identifier Columns (“:” prefix)

Columns that begin with a colon, such as :UniqueId, are identifiers. These columns are used to identify the existing record that should be updated (if it exists) or used to create a new record (if no matching record exists).

This convention allows the system to update the unique identifiers associated with a record. For instance, in a system that uses Unique Id as a unique identifier for a Member Role, a file can change an existing Unique Id like this:

Open

See the individual upload feature for a list of the specific identifiers supported for each file type.

Comment Fields (“#” prefix)

Columns that begin with a hash symbol, such as #Notes, are comments. These columns are ignored by the processing queue.

Typically, the import queue will warn users when they upload a file containing unrecognized columns. This convention allows users to put notes or other metadata into their file without triggering that warning.

Open

Import Queue “Processes”

The final main concept in the Workflow Import Queue is the “Import Process”.

Each “Process” represents a data import operation that a user can use to complete a task. The first thing a user does when using the Import Queue is select the process they want to perform.

Other important concepts for the Import Queue

Identifying the target Workflow Step / Workflow Action

See https://heuristicsolutions.atlassian.net/wiki/spaces/DOCS/pages/3888644139

Conditional Behaviors

An uploaded file might contain data representing numerous types of situations. For instance, some rows might represent “passing” exam scores, others might represent “failing” scores. The actions that the system takes in response to a passing score might be very different than those it takes for a failing score.

These types of scenarios can be implemented using Conditional Behaviors.

Known Limitations