Configure AI features¶

AI features are optional and configured separately for each course. Instructors configure course AI agents and grading behavior, but they can choose only the providers and models enabled by their organization administrator. The course controls appear first below; the final sections explain what instructors should expect when an administrator changes provider availability or usage pricing.

AI settings¶

Open the course-name menu and select AI Settings. Course context and Agreement tolerance are shown first. Advanced settings is collapsed by default and contains AI grading policy and AI agents. We recommend retaining the default policy and agents.

Course context¶

Course context is background used by applicable grading, rubric, solution, and class-performance features. Edit this same course-wide field from Grade → Setup → Exam AI instructions. That card also has separate Grading extra information and Rubric-generation extra information fields for the displayed exam version. See grading setup for scopes, examples, and save controls.

Agreement tolerance¶

Set the Default agreement tolerance independently of the advanced policy. The initial default is 1 point; existing course settings are retained. It is the allowed difference between agent scores in points: with a tolerance of 0.5, scores of 4 and 4.5 agree, while 4 and 5 do not. Select Save agreement tolerance to save it. The default grading configuration uses two agents to assess student work against your rubric and recheck differences.

Advanced settings: AI grading policy¶

Saving policy changes affects future launches; it does not itself change work already running or completed. The separate confirmed reconciliation action below can change eligible existing provisional states.

Grading mode selects non-deliberative independent assessments or deliberative rechecks.
Number of independent agents selects one to three primary models in non-deliberative mode. One model cannot establish AI agreement or automatic provisional approval; this does not prevent a human saving as provisional.
Maximum rechecks permits one or two peer-informed rounds in deliberative mode.
When agents agree, use selects model priority, highest score, or lowest score; model priority breaks ties.

When at least two agents agree, eligible results are automatically saved as Provisional and await human approval. This is not an optional setting and never creates a fully approved grade. Bulk review supports every assessment status; Queue review remains available for individual review. A person can also save an assessment as provisional, with or without AI agreement, or edit it in Bulk review. Human edits do not alter the original AI candidates or policy evidence. See grading states and transitions.

This applies to newly launched runs. Runs already queued retain their recorded settings, and completed results are unchanged. To save eligible older agreed results as provisional, explicitly preview and apply the current policy on the question's Assessment tab.

At the bottom of Generate AI on each question's Assessment tab, Agreement tolerance for this question can override the course default. Leave it blank to inherit the displayed course default. The change saves immediately, affects future AI runs for that question, and refreshes Review and Finalize.

After changing the course default, a question override, or the candidate-selection rule, the question Assessment tab can apply the current provisional-approval policy retrospectively. One confirmed action previews how many existing assessments will be provisionally approved and how many will have provisional status revoked, without making provider calls. The resulting assessments appear as Provisional or Needs review in either workflow. The action does not change historical run outcomes or touch approved, human-authored, human-edited, or stale-rubric assessments. Revoked assessments retain their saved content, history, and all source candidates and scores for full review. See Reconcile provisional status.

Advanced settings: AI agents¶

An AI agent is a named provider-and-model choice available to the course. Agents are course-specific: adding or editing an agent does not change another course.

To add an agent:

Select + Add New Agent.
Enter a unique Agent name for the course.
Choose an LLM provider from the displayed list. If exactly one provider is available, it is selected automatically.
Choose a Model from the displayed list. If exactly one model is available, it is selected automatically.
Select Create Agent.

If the required provider or model is absent, an instructor cannot add it from AI Settings. Contact Gradebird support.

Use Edit on an agent row to change its name, provider, or model. If an agent already has candidate assessments, changing its provider, model, or generation settings moves the original under Show archived agents and creates an updated agent for future work. Other edits are applied directly. Deleting an unused agent removes it; deleting one with prior work moves it under Show archived agents.

Drag the handle at the beginning of an active agent row to change priority. The order saves automatically. Select Show archived agents to inspect archived entries, and use Restore to return an eligible archived agent to active use.

The grading policy uses the first required active agents as primary models. Later active agents are ordered backups. A completed workflow identifies when a backup was used. An archived agent is not selected for new work.

The initial AI configuration is copied when a course is created. Later organization-level changes do not change that course's saved policy or agent order. Review the active agents if an expected choice becomes unavailable.

Review AI-assisted results¶

AI output can propose a rubric, analyze a problem, recognize a marked choice, suggest a roster match, or propose an assessment. It does not publish an exam or rubric, choose the canonical packet, approve a free-response final, or release a student report. Review the visible source evidence and save the human decision required by that workflow.

AI agent prompt preview¶

This read-only page previews the provider-specific prompt structure for an AI agent using representative problem data. It helps an instructor understand how the saved agent configuration is translated for a provider; it does not launch AI work, expose a student response, or modify the agent.

When provider or model availability changes¶

Organization administrators control which providers and models instructors can use. Enabled applies to every AI task using that provider. Turning it off stops grading, imports, clerical work, help, and other AI tasks using it. Requests already sent may finish; subsequent requests are blocked. Saved task assignments and course defaults remain configured but unavailable until the provider is enabled again or reassigned. Disabling with dependent agents, task assignments, or course defaults requires confirmation; it does not require removing those dependencies first. Before an administrator saves an availability change that would retire active course agents, Gradebird shows the number of affected courses, agents, and question assignments. The administrator can select Confirm availability changes to proceed, adjust the available models, or cancel. If the affected records change before confirmation, Gradebird requires review of an updated preview.

Retirement archives the affected agents and removes their question assignments while preserving existing assessments and history. Instructors may need to restore eligible agents or add replacements before generating more assessments. Making a provider or model available again does not automatically restore its agents or assignments: restore each eligible agent and recreate its question assignments.

For an existing provider, Discover Models refreshes the available choices and saves a newly entered, successfully verified API key. Discovery alone does not apply a reduced model selection or retire course agents. Those availability changes occur only after the separate save and, when agents are affected, the confirmation preview.

Failed calls and historical pricing¶

The organization Usage & Cost totals include successful calls and failed requests that reached an AI provider. A timeout or unusable provider response can therefore count as an attempted call. A request rejected locally before provider dispatch does not count as provider usage.

Provider availability is considered restored only after a successful request occurs later than the most recent provider-attempt failure. An earlier success does not clear a newer failure.

When a failed provider request includes usable token or cost information, Gradebird includes that known charge in financial totals. If model pricing was unavailable when the event was recorded, an administrator can add the pricing and run historical repricing; eligible failed calls are repriced along with successful calls, while local denials remain excluded.

Automatic replacement after rubric changes¶

A new agreeing run can replace an assessment from a different rubric, including older human-approved or edited work, with a provisional assessment. Earlier revisions remain in history. Approved and provisional assessments against the same rubric are protected. This differs from retrospective reconciliation, which excludes stale-rubric assessments. See New AI agreement after a rubric change.

Model discovery and compatibility¶

Discovery lists models available to the configured credentials. It does not test whether a model can complete a Gradebird task. OpenAI discovery shows the full returned model list; administrators choose which models to enable. Provider verification does not certify each model for grading.

Choose models that support the task's text, image, and structured-output requirements, and run a small representative task before assigning a model broadly. A listed model intended for another task can fail when selected for grading or import.

Shared AI queue and cancellation¶

AI work shares a fair queue across courses and workflows, including ordinary non-experimental features. The default limit is five executing jobs per user; available worker capacity also limits concurrency. A large grading batch can progress in parallel, while waiting users receive turns as workers become available.

Cancel stops queued work immediately. A running request may need to finish before cancellation completes, and its capacity and reserved usage remain occupied until the executor exits. Waiting alone does not expire queued work. Failed work is not automatically replayed; review the outcome before retrying.