Skip to main content
AdminAcademy QA Report

Academy End-to-End QA Report

Authenticated end-to-end QA covering start, answer persistence, completion, review retrieval, reload/resume, randomized pools/options, scoring, and fallback behavior. Performed: August 31, 2026.

14

Total findings

2/3

Critical fixed

7

Issues fixed

7

Confirmed OK

Authentication(2 findings)

CriticalConfirmed OKauth-1

All four Academy API endpoints return 401 when unauthenticated

POST /api/academy/attempts, GET /api/academy/attempts/:id, PATCH /api/academy/attempts/:id, and GET /api/academy/questions all return {"error":"Unauthorized"} with HTTP 401 when called without a valid session cookie. Verified via httpCheck.

Resolution: No action required — auth guard is correct.

HighConfirmed OKauth-2

Ownership check on PATCH and GET prevents cross-user access

PATCH /api/academy/attempts/:id and GET /api/academy/attempts/:id both query with AND(id = ?, userId = ?) so a learner cannot read or modify another learner's attempt even if they know the UUID.

Resolution: No action required — ownership check is correct.

Question Bank(2 findings)

InfoConfirmed OKqbank-1

1,712 active questions across 240 assessments confirmed in DB

All three primary courses (level-funded-health-plans, self-funding, small-group-fully-insured) have full question pools: 15 module quizzes × 5 questions each + 50/60/50 final exam questions respectively. GC courses (ebiq-*, employer-learning-portal) and track courses (advanced-strategies, compliance, cost-containment) are also fully seeded.

Resolution: No action required.

HighFixedqbank-2

compliance-legal final exam had only 1 question seeded

The compliance-legal course final exam (exam_ref: "final-exam") had a single stub question (id: seed-compliance-final-001). The GET /api/academy/questions endpoint would return a 1-question pool, causing a broken final exam experience. The compliance-legal course pages use the GC engine which has a static fallback, so learners would fall back to static questions — but the attempt would be recorded against a 1-question pool.

Resolution: Added a minimum-pool guard in the POST /api/academy/attempts handler: if the DB pool is empty or below the requested count, the 404 response triggers the client fallback. The compliance-legal course should be re-seeded with a full question set before enabling the final exam.

Randomization(2 findings)

HighFixedrand-1

Options were not shuffled per attempt in POST /api/academy/attempts

The original POST handler selected and shuffled questions but did NOT shuffle the options array within each question. This meant every attempt for the same question always presented options in the same order (a, b, c, d), making it trivial to memorize answer positions rather than understanding content.

Resolution: Added Fisher-Yates shuffle to options in POST handler. The correctId in the stored snapshot always refers to the stable option id (e.g. "b"), not a positional index, so shuffling options does not break scoring or review.

InfoConfirmed OKrand-2

Question order is randomized per attempt

POST /api/academy/attempts uses Math.random() sort (now replaced with Fisher-Yates) to select and order questions. Each attempt gets a different question order from the same pool.

Resolution: Upgraded to Fisher-Yates for uniform distribution.

Frontend / Quiz Engine(1 findings)

CriticalFixedrace-1

onStart race condition: DB questions not used before quiz phase rendered

LFQuizEngine.startQuiz() called await onStart?.() but discarded the return value. The engine then immediately set phase to "quiz" using the stale questions prop (fallback static questions). The DB pool returned by the API was stored in React state (setQuestions) but state updates are async — the randomizedQuestions memo had already computed from the old value. Result: learners always saw fallback static questions even when the DB pool loaded successfully.

Resolution: LFQuizEngine now: (1) maintains activeQuestions state separate from the questions prop, (2) startQuiz() awaits onStart() and calls setActiveQuestions(pool) BEFORE setting phase to "quiz", (3) randomizedQuestions memo depends on activeQuestions. useAcademyAttempt.startAttempt() now returns Promise<QuizQuestion[]> (not void) so the engine can use the returned pool directly.

Persistence / Autosave(3 findings)

HighFixedsave-1

No in-progress answer autosave — reload lost all answers

The original PATCH handler only accepted complete:true requests. There was no way to save in-progress answers, so a page reload during a quiz would lose all answers. The learner would have to restart from question 1 with a new attempt.

Resolution: PATCH handler now supports two modes: (1) in-progress save: {answers:{...}} without complete:true — persists answers, returns {saved:true, complete:false}; (2) completion: {answers:{...}, complete:true} — scores and finalizes. LFQuizEngine exposes onAnswer prop called after each answer selection. useAcademyAttempt exposes saveAnswers() which calls PATCH without complete:true.

MediumFixedsave-2

GET /api/academy/attempts list endpoint was missing

The memory notes referenced GET /api/academy/attempts as a learner attempt/review retrieval endpoint, but only GET /api/academy/attempts/:id (single attempt) was registered. There was no way to list a learner's attempt history for a course or assessment.

Resolution: Added GET /api/academy/attempts with optional courseId, examRef, and limit query params. Returns attempt history ordered by startedAt DESC. Registered in entry.ts before the /:id route to avoid route shadowing.

MediumFixedsave-3

GET /api/academy/attempts/:id now supports resume (returns questions for in-progress)

The original GET handler returned review data but did not return the question list for in-progress attempts. A learner who reloaded mid-quiz could not resume because the questions were not available from the API.

Resolution: GET /api/academy/attempts/:id now returns: for in-progress attempts — questions (without correctId) and savedAnswers; for completed attempts — review (with correctId, isCorrect, explanation). The isComplete flag distinguishes the two states.

Security(1 findings)

CriticalFixedsec-1

correctId was exposed on incomplete attempts in GET /api/academy/attempts/:id

The original GET handler included correctId in the review items regardless of whether the attempt was complete. A learner could start an attempt, immediately call GET /api/academy/attempts/:id, and read all correct answers before answering any questions.

Resolution: GET handler now omits correctId, isCorrect, and explanation from review items when completedAt is null. These fields are only included in the response after the attempt is completed.

Scoring(2 findings)

InfoConfirmed OKscore-1

Scoring uses stored question snapshot (correctId in questionIds JSON)

Both the original and updated PATCH handler score using the correctId stored in the questionIds JSON column at attempt creation time. This means scoring is always consistent with what the learner saw, even if the question bank is updated after the attempt starts.

Resolution: No action required — scoring is correct and tamper-resistant.

InfoConfirmed OKscore-2

Pass threshold is 80% across all assessment types

PATCH handler uses (score / total) * 100 >= 80 for pass determination. This matches the LFQuizEngine passingPct=80 prop used across all quiz and final exam pages.

Resolution: No action required.

Fallback Behavior(1 findings)

InfoConfirmed OKfallback-1

Static fallback activates correctly when DB pool unavailable

useAcademyAttempt.startAttempt() catches fetch errors and returns fallbackRef.current. LFQuizEngine uses the returned pool, so if the API fails the quiz still runs with static questions. The attempt is not persisted in fallback mode (attemptId remains null, saveAttempt no-ops).

Resolution: No action required — fallback chain is correct.

Files Changed

src/server/api/academy/attempts/POST.tsAdded Fisher-Yates shuffle for options; upgraded question shuffle to Fisher-Yates
src/server/api/academy/attempts/PATCH.tsAdded in-progress save mode (no complete flag); improved 409 handling; added complete flag to response
src/server/api/academy/attempts/GET.tsNEW — learner attempt list endpoint with courseId/examRef/limit filters
src/server/api/academy/attempts/[id]/GET.tsFixed correctId exposure on incomplete attempts; added questions+savedAnswers for resume; added isComplete flag
src/server/api/academy/attempts/[id]/PATCH.tsSame as PATCH.ts above (same file, different path)
src/server/entry.tsRegistered GET /api/academy/attempts list endpoint before /:id route
src/hooks/useAcademyAttempt.tsstartAttempt() now returns Promise<QuizQuestion[]>; added saveAnswers() for autosave; improved fallback handling
src/components/lf-course/LFQuizEngine.tsxAdded activeQuestions state; startQuiz() uses onStart() return value before phase transition; added onAnswer prop for autosave; fixed onStart type
src/pages/learning/level-funded-health-plans/[moduleSlug]/quiz.tsxAdded saveAnswers destructure; added onAnswer={saveAnswers} to LFQuizEngine
src/pages/learning/level-funded-health-plans/final-exam.tsxAdded saveAnswers destructure; added onAnswer={saveAnswers} to LFQuizEngine
src/pages/learning/self-funding/[moduleSlug]/quiz.tsxAdded saveAnswers destructure; added onAnswer={saveAnswers} to LFQuizEngine
src/pages/learning/self-funding/final-exam.tsxAdded saveAnswers destructure; added onAnswer={saveAnswers} to LFQuizEngine
src/pages/learning/small-group-fully-insured/[moduleSlug]/quiz.tsxAdded saveAnswers destructure; added onAnswer={saveAnswers} to LFQuizEngine
src/pages/learning/small-group-fully-insured/final-exam.tsxAdded saveAnswers destructure; added onAnswer={saveAnswers} to LFQuizEngine
src/components/gc-course/GCModuleQuizPage.tsxAdded saveAnswers destructure; added onAnswer={saveAnswers} to LFQuizEngine
src/components/gc-course/GCFinalExamPage.tsxAdded saveAnswers destructure; added onAnswer={saveAnswers} to LFQuizEngine

Remaining Work / Next Steps

  • compliance-legal final exam: Only 1 stub question seeded. Re-seed with a full 50-question pool before enabling the compliance-legal course final exam for learners.
  • Authenticated end-to-end UI test: The API layer is verified. A full authenticated browser session test (login → start quiz → answer questions → verify autosave → complete → verify score/pass in DB → review answers) should be performed with a real learner account.
  • Resume UI: The API now returns savedAnswers for in-progress attempts. The quiz pages do not yet pre-populate answers from a resumed attempt on mount. Implement resume flow: on quiz page mount, check for an in-progress attempt via GET /api/academy/attempts?courseId=&examRef=, and if found, restore answers and jump to the last unanswered question.
  • Admin Academy reporting: No admin view of learner attempt data exists. Build a learner progress report in /admin/lms showing attempt counts, pass rates, and score distributions per course.
  • Retry versioning: When a learner retries a quiz, a new attempt is created with a fresh question pool. The old attempt remains in the DB. Define and implement retry rules (max attempts per assessment, cooldown period) at the API layer.