Grade free-response work¶
Prepare rubrics from the Rubric workflow page after publishing the exam. In the Build workflow, the navigation is Build → Print → Scan → Rubric → Grade → Report; printing and scanning are not required to open Rubric. The page supports PDF/Word rubric import, AI suggestions, and individual rubric editing and publication. After reviewing an AI proposal, choose Keep current draft, Replace draft with proposal, or Publish proposal. Publishing selects that rubric for grading while preserving your draft. If its text matches the latest revision, Gradebird reuses and selects that revision, including when an older revision was previously selected. Rubrics remain available from the grading question pages too, using the same drafts and published revisions.
Open Grade¶
Open an exam and select Grade. The workspace is pinned to the displayed Administration version. The grading dropdown starts with Setup and Overview, followed by Auto-graded when applicable and individual questions. Setup contains course context, exam AI instructions, and graders. Overview contains progress, exam-wide queues, recommended actions, and bulk review.
Select a free-response question to open its Question & Rubric and Assessment tabs. Graders reach accessible questions from Grading assignments and see only permitted controls.
Publish a rubric revision¶
In Question & Rubric, edit the rubric draft; draft changes save automatically. Describe how to award points, deductions, caps, and treatment of common errors; do not merely provide a worked solution. Use Preview to check the rendered draft, enter an optional label, and select Publish rubric for the first saved revision or Publish revision for a later one. Publishing creates an immutable numbered revision. Select the canonical revision when more than one exists.
To revise a saved rubric, select its saved revision and choose Edit rubric. Gradebird copies that revision into the editable draft. If a different nonempty draft already exists, confirm that it may be overwritten; an empty or identical draft needs no confirmation. Editing does not change the saved revision or publish a new one. The editor opens without a separate restoration-success notice. Long rendered rubric previews scroll inside their panel.
Rubric AI, when enabled, runs only after an explicit action:
- Propose rubric proposes a rubric from the problem and any optional instructions. View AI proposal opens the result. Replace draft with proposal applies it immediately to the editable draft; Keep current draft dismisses the readiness highlight without deleting the proposal. Publish proposal publishes the proposed text directly for grading without replacing your draft.
- Vet current draft (or the selected published revision) reviews the displayed rubric, reports selective findings, and proposes a minimally changed complete rubric only when findings exist. Applying replaces only the draft. Dismiss potential rubric issues clears the warning highlight for that result while keeping the findings available; a new vetting result can show new warnings.
Generating or vetting a rubric does not publish it automatically. If its inputs change, a completed result is marked out of date. Queued proposals and vetting continue checking for updates while waiting for a worker; ordinary queueing does not require a retry. Empty drafts do not show an unpublished-changes warning, and questions with a published rubric do not show an Empty draft badge.
Changing the canonical rubric moves older approved assessments to Re-review and older provisional assessments to Provisional · Old rubric. Review saved scores and feedback against the new rubric before approving. Approval validates the current rubric without regenerating AI feedback. If the change makes earlier grading unsuitable, revoke approval to return the saved assessment to Needs review; scores, feedback, and history remain available.
Understand question status¶
| Status | Meaning and next action |
|---|---|
| Approved | An instructor or authorized grader approved the final against the current rubric. |
| Re-review | A previously approved assessment uses a rubric revision that is no longer canonical. |
| Provisional | Saved against the current rubric through AI agreement or human action; awaiting human approval. |
| Provisional · Old rubric | Provisionally approved against a rubric revision that is no longer canonical. |
| Needs review | AI models did not agree or a human requested further review. |
| No AI | No AI assessments are available for the current rubric; generate AI candidates or grade manually. |
| Needs rubric | Publish/select a rubric before grading can proceed. |
Grading and Bulk review use the same status counts, ordering, and colors. AI generation is shown separately as activity; it does not replace an assessment’s review status.
Only accepted work from current packets enters the ordinary queues. Resolve scan and packet conflicts first when possible.
Grading setup¶
Choose Setup, before Overview in the grading dropdown, to configure the exam version. The Exam AI instructions card has three independently editable fields: course context, grading extra information, and rubric-generation extra information. Fields show shaded text with an edit button; empty fields show an italic placeholder. Saving returns the field to its read-only display. Use the info button beside each title for scope, guidance, examples, and when changes take effect. Course context affects all exams in the course; the extra-information fields apply only to this exam version. Each extra-information field is added directly after course context in the corresponding prompts for all questions in this version.
Manage graders and question assignments here too. Configure AI grading policy and agents in the course AI Settings workspace. Saved changes affect newly launched AI work.
Find the setup controls¶
The Setup page has two cards in order: Exam AI instructions, then Graders. Check the Administration version above them. Setup is instructor-only; graders use their permitted grading pages.
| Instruction field | Scope and purpose | Example |
|---|---|---|
| Course context | All exams in this course; applicable grading, rubric suggestion and vetting, solution, and performance-analysis prompts. This is the same field as course AI Settings → Course context. | Course level, notation, methods taught, and shared expectations. |
| Grading extra information | AI free-response grading for every question in this exam version. Keep it consistent with each question's rubric. | “Accept equivalent notation; identify the first incorrect step and keep feedback concise.” |
| Rubric-generation extra information | AI rubric suggestions for every question in this exam version, alongside instructions entered for the individual suggestion. | “Include explicit criteria for method, reasoning, and the final answer.” |
Rubric-generation extra information is not sent directly to grading, rubric vetting, or solution generation. The rubric you ultimately publish is used during grading. Grading extra information does not change deterministic auto-grading. To change all students' totals, use Reports → Adjust exam scores….
Select edit beside a field heading to open its text editor. Use Save course context, Save grading information, or Save rubric-generation information to save that field independently and return to the shaded read-only display. Cancel discards that field's unsaved edits. The i beside the title opens scope guidance and examples; close it with × or Escape.
Saving does not regenerate candidates, publish rubrics, or approve grades. Policy grading runs and rubric suggestions retain their launch instructions while queued or running. Clear a field and save it to omit that information from future prompts. Exam-specific guidance is included even when course context is blank. Course copies preserve the exam-specific fields; newly published versions start with empty extra-information fields.
When course-wide grading access is enabled, Graders lists course graders, All questions, and their effective permissions. Follow course Access to invite graders or change that policy. When course-wide access is disabled, this card manages question assignments and their permissions. See Assign graders to questions. Course AI policy and agent order remain in AI Settings; the question-specific tolerance remains on the question's Assessment tab.
Return to progress and review¶
Choose Overview in the same dropdown for progress, recommended actions, the question table, the Auto-graded summary when applicable, and Exam-wide review. Choose an individual free-response question for Question & Rubric and Assessment, or Auto-graded for the combined answer-key, detection, and recognition-review workspace.
Recommended provisional-review links open Bulk review. When both current-rubric and old-rubric provisional assessments exist, the recommendation prioritizes current-rubric provisionals and counts only that group. When only old-rubric provisionals remain, its label includes old rubric and opens that filter. Use Bulk review's status filters to inspect either group explicitly.
Generate AI candidates¶
The default controls use the saved course policy and active agent order:
- Generate subset launches fresh runs for the inclusive from/to positions; the range is saved per user and question.
- Generate remaining targets responses with no completed candidate assessment for the current rubric, also skipping responses with saved Approved or Provisional assessments against the current rubric, even when they have no AI candidate. Re-review and Provisional · Old rubric remain eligible when current-rubric candidate coverage is missing. Revoked assessments are eligible when that coverage is missing too.
- Generate all launches fresh runs for every accepted response and confirms when existing candidates are present.
Each generation uses the course policy and rubric in effect when it starts. Later configuration changes affect future work only.
AI candidates are proposals, including when agents agree. New runs automatically save eligible agreed results as Provisional, awaiting human approval; both Bulk review and Queue review remain available. Already queued runs keep their launch settings, and existing results are unchanged unless explicitly reconciled. A one-model run cannot automatically provisionally approve; a human can still save an assessment as provisional. See Configure AI features to change the course policy.
New AI agreement after a rubric change¶
A new agreeing run for the current canonical rubric can replace the saved assessment from a different rubric. This includes older approved, human-authored, and human-edited assessments. The new current assessment is Provisional, needs human approval, and retains earlier assessment revisions in history.
Within the same rubric, an existing approved or provisional assessment is protected from automatic replacement; a revoked assessment remains eligible. If the canonical rubric changes again before the run saves its result, that run cannot provisionally approve against its now-outdated rubric. Changing the canonical rubric alone does not restore a historical assessment or create a new one.
For example, an approved assessment under revision 1 becomes Re-review when revision 2 is made canonical. You can review and approve the saved assessment against revision 2. Alternatively, new AI grading under revision 2 can replace it with a provisional assessment if the agents agree. This differs from retrospective policy reconciliation below, which excludes stale-rubric assessments.
Saved assessment states and transitions¶
The saved assessment, its approval state, the current rubric, and the AI candidates are separate things. A saved score does not by itself mean a grade is approved. The name Assessment refers to the saved grading decision even when it is provisional or revoked.
| Saved state | Review behavior | Counts as a current approved grade? |
|---|---|---|
| No saved assessment | No AI offers manual entry in Bulk review. For Needs review with AI candidates but no save, use Queue review to compare and copy a candidate. Unsaved/copied text is not a saved assessment. | No. |
| Provisional, current rubric | Review and edit in either workflow, then approve. May originate from AI agreement, human editing, or human authorship. | No. |
| Provisional · Old rubric | Review against the current rubric; save as provisional, approve, or revoke. | No. |
| Approved, current rubric | Review or edit in either workflow; saving as provisional removes final approval. Bulk review can revoke approval directly. | Yes, within the applicable packet/report scope. |
| Re-review | Historical approval is retained, but approval against the current rubric is required. | No. |
| Revoked | Always Needs review, even without AI candidates. Saved score, assessment, source, and revisions are retained and can be saved as provisional or approved again. | No. |
| No canonical rubric | Publish/select a rubric before saving. This is a prerequisite, not an approval state. | No. |
| Action | Result |
|---|---|
| Queue review: Save and continue → as approved; Bulk review: Save and approve | Saves the entered score/text and approves against the current rubric. |
| Queue review: Save and continue → as provisional; Bulk review: Save as provisional | Saves against the current rubric for further review, removing any final approval. |
| Bulk review: Approve | Approves the eligible saved assessment against the current rubric without changing its score/text. |
| Bulk review: Provisionally approve | In a Needs review query, saves eligible revoked assessments as provisional against the current rubric without changing score/text. Unsaved AI candidates cannot be bulk-approved. |
| Revoke approval | Removes approved or provisional status and returns the saved assessment to Needs review, retaining content and history. |
| Copy an AI candidate, edit without saving, or change checkbox selection | Does not change the saved assessment or approval state. |
| Save identical content, source, rubric, and approval state | No new revision. A rubric or approval-state change still counts as a change even if score/text are identical. |
Removing approval affects current grading/report readiness, not already released report snapshots. Scores must be between zero and the published question maximum. Missing-work assessments can also be reviewed; inspect any unavailable-work warning before saving or approving.
Reconcile provisional status¶
The course Default agreement tolerance applies to every question that does not have an override. At the bottom of Generate AI on a question's Assessment tab, an instructor can enter Agreement tolerance for this question; leaving it blank uses the displayed course default. The value saves when changed, applies to future runs for that question, and refreshes Review and Finalize. Existing runs retain the effective tolerance recorded when they launched.
After changing the course default, a question override, or the candidate-selection rule, an instructor can use the question's Assessment tab to apply the current provisional-approval policy to existing AI results. The panel previews how many assessments will be provisionally approved and how many will have provisional status revoked. Its confirmed action can perform both transitions together and makes no AI provider calls. The resulting assessments appear as Provisional or Needs review in either workflow.
For an unsaved response, Gradebird evaluates the latest completed, coherent policy run for the current canonical rubric. If its source candidates agree under the current tolerance, Gradebird selects the candidate using the current consensus rule and saves it as provisional. If they do not agree, the response remains in normal human review.
For an existing provisional assessment, Gradebird evaluates the completed current-rubric run containing its source candidate. If those source candidates no longer agree under the current tolerance, Gradebird removes provisional approval. This removal applies only when Gradebird originally created the provisional assessment, its saved score and assessment still match the AI source, and its history contains no human content edit. Approved, human-authored, human-edited, and stale-rubric assessments are not changed.
Each evaluation records the current policy and its pre_approved, not_agreed, or revoked decision separately. The original run keeps its original agreement outcome and launch-time tolerance. A revocation revision identifies the instructor who confirmed the action. The assessment's saved score, feedback, source, and history remain available; full review shows every source candidate and score so the grader can compare them before saving as approved or provisional.
You can reconcile again after changing tolerance back to a previously used value. Gradebird checks the current saved assessment state and preserves the earlier decisions in its history. Repeating the same action without an intervening assessment change does not create another decision. If someone edits or approves an assessment while reconciliation is being applied, that assessment is skipped; refresh the Assessment tab to review the current state.
Queue review: finalize one response¶
Choose a status queue or an individual segment in the status bar. The review page shows response evidence, Candidates, Rubric, and Question reference tabs, and the final Assessment editor.
- Inspect every accepted response crop for the target question.
- Read the canonical rubric and published question context.
- Open candidate details as useful. Copy to assessment copies a proposal's score and student-facing assessment text into editable final fields; it does not approve automatically.
- Enter a score from zero through the question maximum and edit the assessment LaTeX.
- Choose as approved (the default) or as provisional beside Save and continue. As approved saves and approves. As provisional saves for further review without final approval; if the assessment was already approved, this removes that approval. The queue advances according to the selected review mode.
Full review keeps a short, friendly Gradebird note above the Candidates / Rubric / Question tabs, so switching tabs never hides it. The bold heading highlights approved assessments or human-edited provisional work, and otherwise describes the AI candidates; smaller guidance underneath explains the saved assessment's state and what to do next. It contains no names, scores, timestamps, or revision details. Expand the muted Assessment history disclosure at the bottom of the middle Assessment card to inspect who saved each revision, changes to score or text, its rubric and source candidate, and whether it was saved as approved, saved as provisional, or had approval removed. The friendly note and history summary are assembled from recorded state, not generated by an AI model. Original AI candidates remain available in the Candidates tab.
For approved assessments under the current rubric, the heading is This assessment has been approved. If the rubric has since changed, the heading is This assessment was approved under an earlier rubric. The subtext uses recorded provenance to explain whether an AI candidate was adopted unchanged, edited by a human before approval, or the assessment was written by a human. If its origin cannot be established, it refers to Assessment history without assuming an AI source.
The exact approved subtext is chosen from the saved record:
| Recorded approval provenance | Exact subtext |
|---|---|
| The saved score and assessment text match the linked AI candidate | An AI candidate was adopted and approved by a human. |
| A linked AI candidate exists, the saved score/text differ, and the history records human content editing | An AI candidate was edited by a human before the assessment was approved. |
| Human authorship/editing is recorded and there is no linked source candidate | This assessment was written by a human and approved. |
| The available record does not establish one of those origins | A human approved the saved assessment. See Assessment history for the recorded revisions. |
The adopted-recommendation message describes the currently saved content. If earlier edits were later restored to match the source candidate, that message can appear; Assessment history retains the intervening revisions. It does not mean that every AI model agreed, or that no earlier edits occurred.
For human-authored or human-edited provisional assessments, the heading is This assessment needs final approval. This takes priority over AI candidate status, including agreement. The same heading is used when a human selected an AI candidate after the agents did not agree. Original AI candidates remain available in the Candidates tab.
Retrospective policy history also takes priority over the original AI-run heading when it explains the current assessment state:
| Recorded history | Heading | Guidance |
|---|---|---|
| A disagreed run was provisionally approved after the tolerance changed | This assessment was provisionally approved under updated settings. | The agents’ scores were outside the tolerance used for the original AI run, but they fell within the tolerance applied later. Review the selected assessment and save as approved. |
| An agreed run was provisionally approved by applying the policy later | This assessment was provisionally approved under updated settings. | The agents agreed under the original AI-run settings. The provisional-approval policy was applied later to save the selected assessment for review. Review it and save as approved. |
| An originally agreed provisional assessment was revoked retrospectively | This assessment’s provisional status was revoked under updated settings. | The agents’ scores were within the tolerance used for the original AI run, but they fell outside the tolerance applied later. The saved assessment is still available; review it, edit if needed, and save. |
| A retrospective provisional approval was itself revoked by a later policy application | This assessment’s provisional status was revoked under updated settings. | Earlier updated settings made this assessment eligible for provisional approval, but it was no longer eligible under the settings applied most recently. The saved assessment is still available; review it, edit if needed, and save. |
| An originally agreed unsaved result was later evaluated as outside tolerance | The agents agreed under the original settings. | Their scores fell outside the tolerance applied later. Review the candidates or write an assessment, then save as approved. |
| Saved state | Guidance below the heading |
|---|---|
| No saved assessment; current-rubric candidates available | Review the candidates or write an assessment, then save as approved. |
| No saved assessment; no current-rubric candidates available | Write an assessment, or generate AI candidates, then save as approved. |
| Provisional assessment without human edits | Review the provisional assessment, edit if needed, and save as approved. |
| Human-authored or human-edited provisional assessment | This provisional assessment includes human edits. Review it and save as approved. |
| A human selected a candidate from a disagreed result and saved it provisionally | A human selected an AI candidate after the agents did not agree. Review it and save as approved. |
| New generation is running while an unchanged provisional assessment already exists | New AI assessments are being generated. The saved provisional assessment still needs human review and approval. |
| Approved assessment | Explains whether a human adopted an AI candidate, edited it before approval, or wrote the assessment. |
| Revoked assessment | The saved assessment is still available; review it, edit if needed, and save. |
When a saved assessment uses a different rubric, its guidance adds Review against the current rubric before saving. An approved assessment with that warning requires re-review; its historical approval does not count as a current-rubric approval. When no rubric is available, saving is unavailable and the page shows the applicable rubric waiting/setup state. The guidance text, where rendered, is A published rubric is needed before you can save an assessment.
| AI candidates | Bold heading |
|---|---|
| Generation in progress | Generating AI assessments. |
| Current candidates have an agreed result | The agents agreed on a score. |
| Unresolved deliberation | The agents did not agree. |
| Generation failed but candidates are available | AI grading was incomplete. Available candidates can be reviewed. |
| Generation failed and no candidate is available | AI grading failed. No candidate available. |
| Multiple candidates without an agreement result | Multiple AI assessments available. |
| One candidate | One AI assessment available. |
| No candidates | No AI assessment yet. |
Heading precedence is: retrospective history that explains a current provisional/revoked state; approved assessment; human-authored, human-edited, or human-selected provisional assessment; an existing unchanged provisional assessment while newer generation runs; then the AI-candidate rows above in their listed order. Thus a saved status is not hidden by newer generation. A rubric mismatch appends the review reminder to the subtext; for approved work it also changes the heading to identify approval under an earlier rubric.
A non-retrospective revocation uses This assessment’s provisional status was revoked. when the prior revision was provisional, This assessment’s final approval was revoked. when the prior revision was approved, and This assessment’s approval was revoked. when the available history does not establish the prior state. Bulk review can revoke current or older-rubric approval, as well as provisional approval. Revocation never means the saved assessment was withdrawn or deleted.
When provisional approval was removed by retrospective policy reconciliation, use Reconcile provisional status for the preserved-history and candidate-comparison details.
Candidate descriptions refer to the current rubric's candidates, not to subsequent human edits of the saved assessment. For assessments that are neither approved nor human-edited provisional work, generation in progress takes priority; otherwise an available agreement/disagreement result takes priority over a failure notice. These descriptions use fixed text, not an AI-generated narrative.
Candidate comparison is separate from the fixed note and does not approve grading. With at least two eligible current-rubric candidates, comparison generation may start automatically when the review panel loads. Gradebird is thinking... indicates it is in progress; a completed See summary opens Assessment comparison, containing Summary and Recommendation. A failed comparison offers Try again when eligible; unmet comparison prerequisites can disable See summary. With too few candidates and no existing result, there may be no comparison control. Closing the comparison changes no saved assessment. The fixed note stays outside the reference tabs and refreshes when candidate information updates.
Revoking approval retains the latest saved score and assessment in full review, including human edits. Saving that assessment as provisional returns it to provisional review. Saving it as approved approves it. A provisional assessment may be human-authored or human-edited; provisional does not necessarily mean the AI models agreed on the current text or score.
Full-review saves validate the displayed saved assessment, canonical rubric, and accepted work. If any changed while you were reviewing, Gradebird rejects the stale save instead of overwriting the newer state. This also applies when two reviewers save the first assessment for the same response or missing-work item: the first save is retained and the other reviewer must reload before saving. Your submitted fields remain visible with an error and Reload saved assessment. Copy any unsaved text you want to retain before reloading; review the latest content, rubric, and evidence before submitting again. Invalid scores also keep the form open: scores must be nonnegative and no greater than the published question maximum.
Assessment history is collapsed by default and ordered newest first. Each revision exposes the saving actor (or Gradebird when no human actor is recorded), timestamp, score, rubric revision, approval state, linked source candidate when present, and expandable saved assessment text and notes. The summary identifies the original linked AI candidate and latest recorded content editor when available. Human content editing is distinct from an approval-only or revocation-only revision. Provisional saves record the human editor without identifying them as a final approver. Original AI candidate content and historical policy evidence are retained.
Missing-work assessments support both as approved and as provisional in full review. Use the missing-work path when the current packet should receive a score without an accepted response. If work is scanned later, Gradebird asks you to resolve the conflict before replacing that grading decision.
Candidate thumbs-up/down feedback records quality feedback for Gradebird review; it does not change the grade.
Bulk review¶
Bulk review handles multiple responses in any assessment state. It is available to everyone with grading access, for their permitted questions; no experimental setting is required. Queue review handles one response at a time and includes AI candidate comparison and Copy to assessment.
Open and filter results¶
Enter bulk review opens the status bar and counts without loading cards. Select a status in the bar, or choose a question, status, order, and page size and select Show results. Recommended actions already choose a relevant queue and load it: provisional and re-review work use Bulk review; Needs review uses Queue review. No AI recommends generation when appropriate, otherwise manual grading. Both the exam and question pages are titled Bulk review.
Choose all questions or one question. Results group by question and sort by bird name, student name when authorized (last name, then first name; unmatched packets last), or saved score in either direction. Unsaved assessments sort last for score ordering; AI candidate scores are not used. Page sizes are 20, 50 (default), 100, or 200. The Results card identifies the applied question/status, Total count, page, and order. Pagination appears at the top and bottom when needed. Empty results show a simple message. After a bulk action clears the selected category, you return to the page you came from. If records were skipped because they changed, you remain in Bulk review to inspect the result. Changing controls takes effect when you select Show results; pagination preserves the applied query.
Review or edit a card¶
Each card shows its permitted bird/student identity, status, question/points, rubric revision, student work, and any saved score and assessment. Saved assessments have a shaded box. Drag the divider to adjust the work and assessment widths independently on each card; smaller screens stack the panels. Identity display and name sorting remain subject to your permissions.
For any saved assessment, Edit offers Save as provisional and Save and approve, both against the current rubric. Muted Approve and Revoke approval links share a row with provenance directly beneath the assessment, stay in place when history expands below, and ask for confirmation. No AI cards have optional empty manual fields and a Return to grading link to generate AI assessments.
When a Needs review card has AI candidates but no saved assessment, its Queue review link opens that response in the question’s Needs review queue so you can compare candidates and copy one into the assessment. Cards without AI retain the link back to grading.
Saving updates the card in place and retains its checkbox selection. Its new status may differ from the fetched Results filter: the card stays visible until results are refreshed, and the Total count stays unchanged. Bulk-action counts update to reflect saved state changes. Unsaved edits do not block navigation or individual card actions; leaving or replacing a card discards them. Fetch/save spinners temporarily disable conflicting controls.
Select and apply bulk actions¶
Every saved assessment starts checked, including approved items; cards without a saved assessment have no checkbox. Select all and Unselect all affect the current page only. Rubric publication clears selections so you can review the refreshed cards before selecting again. Bulk conflicts instead flag the affected cards without replacing them.
The Bulk actions menu starts closed and offers two scopes:
| Scope | Affected assessments |
|---|---|
| Selected on this page | Eligible checked saved assessments on the displayed page. |
| All results on all pages · ignores checkboxes | Eligible saved assessments from the fetched result set across every page, regardless of selection. Later arrivals and other question/status results are excluded. |
Approve is available for saved assessments not already approved against the current rubric. Revoke is available for approved or provisional saved assessments. In a Needs review query, Provisionally approve is also available for saved assessments still in Needs review. Menus and counts reflect current card states, including inline changes; unsaved candidates are never approved implicitly.
Every menu action confirms its count, question/all-question scope, and effect. A warning appears when affected assessments extend beyond the displayed page. Approving or provisionally approving off-page assessments additionally requires checking that you reviewed all the AI assessments. Revocation always retains saved scores, feedback, and history and returns items to Needs review.
Provisional, Provisional · Old rubric, Re-review, and Needs review queries also have a primary action at the bottom when saved cards are present. It approves eligible checked page items without another confirmation. Provisional and re-review queries offer Also revoke unselected, unchecked by default; enable it to revoke eligible unchecked page items in the same submission. Otherwise unchecked items stay unchanged. Needs review has no footer revocation option. The footer mode stays tied to the fetched query even when individual cards change state. Bulk status actions use the saved assessment. If you have changed any assessment score or feedback, save or undo those edits before submitting a bulk action. Merely opening an editor, selecting cards, or leaving a blank manual assessment untouched does not block submission. This check applies only to bulk actions; it does not add navigation warnings. After submission, the page reloads with the same filters and page number (or the last remaining page). Approving page 1 therefore brings the next remaining matching responses onto page 1; an empty queue shows the empty-results message.
Show or revise the rubric¶
Select Rubric rN to show the current question and rubric alongside the work and assessment. Wide screens add a third column with its own divider; narrow screens show the reference above the work. The reference is preloaded, so show/hide requires no further fetch. Old-rubric provisional pills identify the saved revision, such as Provisional · Old rubric (r2); the Re-review tooltip explains the saved/current revisions and what approval validates.
With rubric-edit permission, Edit rubric opens the publication dialog. Close it with its close control; clicking outside does not dismiss it. Successful publication closes the dialog, refreshes results, and clears selections without a success notice. Review the updated assessments before continuing.
Inspect provenance and handle changes¶
The muted Provenance: AI / Edited / Human / Unknown label summarizes the saved assessment's origin. Show details loads its full history on demand, newest first, distinguishing AI auto-saves, named human edits, approvals, and revocations. Names fall back to email when necessary. The AI origin revision contains the available View full AI deliberation link, opening in a new tab; later revisions do not repeat the original AI details. Approval alone is not a human content edit.
Opening Bulk review does not lock records while you read. Changes elsewhere are checked again when you act:
- Saving an edit: a conflict rejects the save and keeps your submitted fields visible. Before selecting Reload review, copy unsaved scores and feedback somewhere safe: reloading replaces the cards and discards all unsaved edits.
- Bulk status actions: unchanged assessments are processed; those already in the requested state are left unchanged; changed, missing, or unavailable assessments are skipped. One conflict does not block the rest. If any assessments were skipped, a dismissible warning reports each count, for example “49 approved; 1 skipped,” including skips on other pages. This warning appears above Results and lists each affected bird/packet and question, the skip reason, its current status, and a link to review it—even if it moved outside your filters. Removed or inaccessible items may have no review link. Successful batches reload results without a success notice. Bulk errors and unsaved-edit warnings are also dismissible; dismissing a warning does not bypass its underlying check.
- After a bulk action: results reload from the latest saved state. Skipped assessments remain in the results if they still match your filters; review them again before retrying. An all-results action processes its original scope across every page, then refreshes the current page. Newly arriving assessments are not included in that action but may appear in the refreshed results.
Invalid or expired review scopes and missing off-page review confirmation reject the request without changing assessments. A database or audit failure can still prevent the operation as a whole; it is not silently treated as a skipped item. Approval/revocation history records only changes actually made. Active AI generation is shown separately: automatic AI saves may replace revoked assessments, but preserve saved provisional or approved assessments.
AI solution and candidate comparison¶
The AI Solution panel asks the current primary grading models to solve the published problem independently. Changes to the problem, course context, or primary-agent selection make the prior solution review out of date. Disagreement or serious issues are advisory and do not validate or invalidate the rubric automatically.
Solution mathematics is rendered with KaTeX. Exam front matter, preamble packages, and custom command definitions apply only to the compiled exam PDF; they are not loaded into solutions, rubrics, assessments, or report previews. Use standard LaTeX commands that KaTeX supports, expand custom shorthand into those commands, and check the rendered preview. A successful exam PDF preview does not establish compatibility with these browser views.
Candidate summaries compare assessments when at least two current-rubric candidates exist. Instructors may archive a candidate from the current view and restore it later.
Completion boundary¶
Unsaved assessments, provisional assessments, revoked assessments, and stale approved finals do not make grading release-ready. Every gradeable free-response question on the canonical packet must have a current approved final. Auto-graded questions separately require a current definitive recognition and score against the current key. Then continue to Reports and exports.
Grading assignments¶
This page lists accessible questions from each exam’s selected administration version, through course-wide access or per-question assignments. Publishing another version does not add duplicate rows: the instructor’s selected version controls which questions appear. Exams without a selected canonical version are omitted. Each row summarizes remaining review work and links to the permitted question workspace or grader overview. The page does not grant broader course access than the effective course or assignment permissions.
Grading overview¶
This grader-facing overview summarizes only the questions accessible to the current grader for one exam version. It provides question navigation, progress, and recommended next actions while respecting rubric-edit and student-name permissions.
Manage grades¶
The instructor overview coordinates question selection, rubric readiness, auto-graded work, review queues, and exam-wide progress for the administration version. Use it to choose the next grading action; counts and recommendations do not themselves approve or change grades.
Grade a question¶
This workspace manages one published question’s rubric and assessment workflow. Depending on permission, an instructor or grader can inspect the question, publish or select a rubric revision, launch eligible AI assistance, review queue state, and move into individual or bulk review. A selected tab is local page state and does not change the question identity. Generate remaining skips current-rubric Approved and Provisional assessments, even without AI candidates. Re-review and Provisional · Old rubric remain eligible if current-rubric AI coverage is missing.
Queue review¶
This page reviews the accepted scan evidence, current rubric, AI candidates, provisional state, and final assessment for one response. Authorized users can generate eligible candidates, inspect provenance, select or edit an assessment, save as approved or provisional, or navigate the active queue. Saving as approved approves; saving as provisional preserves work for further review without approval. Both changed saves are recorded in revision and application audit history.
Bulk review¶
Review saved assessments across the exam version, with question and status filters, inline editing, provenance, and selected or all-results bulk actions. Only questions you are authorized to grade are included. Old bulk-review links redirect here.
Bulk review¶
The same review interface has one published question preselected; the generic entry shows counts but waits for a status click or Show results before loading cards. Its bulk actions retain the displayed question scope and the same permission, confirmation, and concurrency checks.
AI solution details¶
This detail page shows the outcome and provenance of an instructor-requested AI solution check for a published question. It can explain agreement, warnings, and the models involved, but the check remains advisory and cannot publish content or approve grading.
AI assessment details¶
This page shows one candidate assessment, its score, assessment text, notes, and available provenance. Instructors use it to compare candidate reasoning with the rubric and response evidence; viewing a candidate does not make it final.
AI grading run¶
This page summarizes one saved-policy grading run, including attempts, selected candidate, and the policy snapshot used when it started. It is historical evidence: later course or agent changes do not rewrite the run.
Deliberative grading run¶
This page presents the rounds and recommendation from a historical deliberative grading run. It supports review of existing records but does not expose the developer/tester launch controls as a supported instructor workflow.
Reuse rubrics after republishing an exam¶
Publishing a new exam version does not automatically carry rubrics forward. At the top of the Rubric page, Use rubrics from a previous version? offers reuse while questions lack published rubrics. Select Review and import previous rubrics to see eligible matches. Gradebird also automatically offers eligible imports on instructor grading overview and question pages. The offer lists each current question, its source exam version and question number, and the saved rubric revision. Expand Preview rubric to read the text, then select Import rubrics to confirm the whole list.
Each question must meet all of these conditions:
- It is a free-response question with no saved rubric revisions and an empty rubric draft. Clearing the draft does not make a question eligible if it already has saved revisions.
- It matches a question with a saved rubric in an earlier version of the same exam. The question number can differ.
- The question text (including LaTeX and whitespace), points, question metadata, parent and sibling subquestion text, response-box configuration, referenced image contents, and exam LaTeX preamble match exactly. Exam title and front-matter changes, such as editing “Good luck!”, do not prevent a match. The internal PDF template ID is ignored, so publishing a revised PDF does not by itself prevent reuse.
For each match, Gradebird uses the latest saved rubric revision from the newest earlier exam version that has one. Imports create independent revision 1 rubrics with empty labels and select them for grading in the current version. Existing drafts and saved rubrics are preserved.
Close stops automatic offers for you on that exam version, including on other question pages. While eligible rubrics remain, you can still open the Import rubrics from a previous exam version card near the bottom of the grading overview. The notice at the top of the Rubric page remains available regardless of this Close preference.
No eligible rubrics means no overview import card or automatic offer. The Rubric page still explains why reuse is unavailable if questions lack published rubrics, including existing drafts, changed questions, or no earlier saved rubrics. Its notice disappears once all free-response questions have published rubrics. If a preview expires or rubric work changes before confirmation, review the refreshed list and submit again.