7: Testing and Debugging

Last updated
Save as PDF

Page ID: 555731

$ \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } $

$ \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} $

$ \newcommand{\dsum}{\displaystyle\sum\limits} $

$ \newcommand{\dint}{\displaystyle\int\limits} $

$ \newcommand{\dlim}{\displaystyle\lim\limits} $

$ \newcommand{\id}{\mathrm{id}}$ $ \newcommand{\Span}{\mathrm{span}}$

( \newcommand{\kernel}{\mathrm{null}\,}\) $ \newcommand{\range}{\mathrm{range}\,}$

$ \newcommand{\RealPart}{\mathrm{Re}}$ $ \newcommand{\ImaginaryPart}{\mathrm{Im}}$

$ \newcommand{\Argument}{\mathrm{Arg}}$ $ \newcommand{\norm}[1]{\| #1 \|}$

$ \newcommand{\inner}[2]{\langle #1, #2 \rangle}$

$ \newcommand{\Span}{\mathrm{span}}$

$ \newcommand{\id}{\mathrm{id}}$

$ \newcommand{\Span}{\mathrm{span}}$

$ \newcommand{\kernel}{\mathrm{null}\,}$

$ \newcommand{\range}{\mathrm{range}\,}$

$ \newcommand{\RealPart}{\mathrm{Re}}$

$ \newcommand{\ImaginaryPart}{\mathrm{Im}}$

$ \newcommand{\Argument}{\mathrm{Arg}}$

$ \newcommand{\norm}[1]{\| #1 \|}$

$ \newcommand{\inner}[2]{\langle #1, #2 \rangle}$

$ \newcommand{\Span}{\mathrm{span}}$ $ \newcommand{\AA}{\unicode[.8,0]{x212B}}$

$ \newcommand{\vectorA}[1]{\vec{#1}} % arrow$

$ \newcommand{\vectorAt}[1]{\vec{\text{#1}}} % arrow$

$ \newcommand{\vectorB}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } $

$ \newcommand{\vectorC}[1]{\textbf{#1}} $

$ \newcommand{\vectorD}[1]{\overrightarrow{#1}} $

$ \newcommand{\vectorDt}[1]{\overrightarrow{\text{#1}}} $

$ \newcommand{\vectE}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash{\mathbf {#1}}}} $

$ \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } $

$\newcommand{\longvect}{\overrightarrow}$

$ \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} $

$\newcommand{\avec}{\mathbf a}$ $\newcommand{\bvec}{\mathbf b}$ $\newcommand{\cvec}{\mathbf c}$ $\newcommand{\dvec}{\mathbf d}$ $\newcommand{\dtil}{\widetilde{\mathbf d}}$ $\newcommand{\evec}{\mathbf e}$ $\newcommand{\fvec}{\mathbf f}$ $\newcommand{\nvec}{\mathbf n}$ $\newcommand{\pvec}{\mathbf p}$ $\newcommand{\qvec}{\mathbf q}$ $\newcommand{\svec}{\mathbf s}$ $\newcommand{\tvec}{\mathbf t}$ $\newcommand{\uvec}{\mathbf u}$ $\newcommand{\vvec}{\mathbf v}$ $\newcommand{\wvec}{\mathbf w}$ $\newcommand{\xvec}{\mathbf x}$ $\newcommand{\yvec}{\mathbf y}$ $\newcommand{\zvec}{\mathbf z}$ $\newcommand{\rvec}{\mathbf r}$ $\newcommand{\mvec}{\mathbf m}$ $\newcommand{\zerovec}{\mathbf 0}$ $\newcommand{\onevec}{\mathbf 1}$ $\newcommand{\real}{\mathbb R}$ $\newcommand{\twovec}[2]{\left[\begin{array}{r}#1 \\ #2 \end{array}\right]}$ $\newcommand{\ctwovec}[2]{\left[\begin{array}{c}#1 \\ #2 \end{array}\right]}$ $\newcommand{\threevec}[3]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \end{array}\right]}$ $\newcommand{\cthreevec}[3]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \end{array}\right]}$ $\newcommand{\fourvec}[4]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}$ $\newcommand{\cfourvec}[4]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}$ $\newcommand{\fivevec}[5]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}$ $\newcommand{\cfivevec}[5]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}$ $\newcommand{\mattwo}[4]{\left[\begin{array}{rr}#1 \amp #2 \\ #3 \amp #4 \\ \end{array}\right]}$ $\newcommand{\laspan}[1]{\text{Span}\{#1\}}$ $\newcommand{\bcal}{\cal B}$ $\newcommand{\ccal}{\cal C}$ $\newcommand{\scal}{\cal S}$ $\newcommand{\wcal}{\cal W}$ $\newcommand{\ecal}{\cal E}$ $\newcommand{\coords}[2]{\left\{#1\right\}_{#2}}$ $\newcommand{\gray}[1]{\color{gray}{#1}}$ $\newcommand{\lgray}[1]{\color{lightgray}{#1}}$ $\newcommand{\rank}{\operatorname{rank}}$ $\newcommand{\row}{\text{Row}}$ $\newcommand{\col}{\text{Col}}$ $\renewcommand{\row}{\text{Row}}$ $\newcommand{\nul}{\text{Nul}}$ $\newcommand{\var}{\text{Var}}$ $\newcommand{\corr}{\text{corr}}$ $\newcommand{\len}[1]{\left|#1\right|}$ $\newcommand{\bbar}{\overline{\bvec}}$ $\newcommand{\bhat}{\widehat{\bvec}}$ $\newcommand{\bperp}{\bvec^\perp}$ $\newcommand{\xhat}{\widehat{\xvec}}$ $\newcommand{\vhat}{\widehat{\vvec}}$ $\newcommand{\uhat}{\widehat{\uvec}}$ $\newcommand{\what}{\widehat{\wvec}}$ $\newcommand{\Sighat}{\widehat{\Sigma}}$ $\newcommand{\lt}{<}$ $\newcommand{\gt}{>}$ $\newcommand{\amp}{&}$ $\definecolor{fillinmathshade}{gray}{0.9}$

This chapter helps you find and fix problems in your WeBWorK questions. Whether you are setting up local rendering for the first time, debugging a broken problem, or doing a final check before publishing, start here to find the right section.

Debugging WeBWorK problems is not fundamentally different from troubleshooting a lab protocol. You isolate the variable that changed, reproduce the failure under controlled conditions, and fix one thing at a time. The tools in this chapter give you a fast feedback loop so you spend less time guessing and more time writing good questions.

What just went wrong?

Use this table to jump straight to the section that matches your symptom. If you are not sure what went wrong, start at the top and work down.

What just went wrong?: what you see, go to, and what you will find.
What you see	Go to	What you will find
Render fails with an error message	Common Mistakes and How to Fix Them	Common mistakes organized by symptom with copy-paste fixes
Widget shows as a plain text box instead of radio buttons or dropdown	Common Mistakes and How to Fix Them	Variable scoping rules (the `my` keyword trap)
HTML tags appear as literal text in the output	Common Mistakes and How to Fix Them	The `[$var]*` rule for HTML variables
Problem works for one seed but breaks on others	Testing Randomization and Edge Cases	Randomization testing and guard patterns
Everything looks fine but want a final checklist	QA Checklist Before Publishing	QA checklist before publishing
Need to lint or test many files at once	Scripting and Automation	Scripting and batch automation
Setting up local rendering for the first time	Setting Up the PG Renderer	Install, start, and first render walkthrough

Section map

Section map: section and what it covers.
Section	What it covers
Setting Up the PG Renderer	Install the renderer and confirm it works with your first problem file.
Common Mistakes and How to Fix Them	The big reference. Symptom-based lookup for PGML parsing, variable scoping, HTML escaping, macros, and more.
Simple Syntax Checking (Linting)	Static checks you can eyeball and renderer-based lint that catches issues before students see them.
Testing Randomization and Edge Cases	Sweep seeds to find broken variants, guard against impossible values.
QA Checklist Before Publishing	A prevention checklist and author checklist to run before you ship.
Scripting and Automation	API endpoints, Python examples, and batch linting for advanced workflows.

Start here

If you are new to local rendering, start with Setting Up the PG Renderer. It walks you through installing the renderer and confirming your first problem file works.
If something just broke, go to Common Mistakes and How to Fix Them. It is organized by symptom so you can look up what you see and find a fix without reading everything else first.
If you are about to publish, go to QA Checklist Before Publishing. The QA checklist catches the problems that look fine in author view but fail when students encounter them.

Apply it today

Use the decision table above to jump directly to the section that matches your current situation instead of reading the chapter front to back.
Keep the QA checklist from QA Checklist Before Publishing open as a tab while you finalize problems for a new assignment.

7.1: Simple Syntax Checking (Linting) in WeBWork
Covers static checks you can do by eye (PGML tag wrappers, HTML without *, MODES in eval blocks, blocked HTML tags) and renderer-based lint using lint_pg_via_renderer_api.py with -r and -s flags. After finishing, you can run a quick lint pass on any problem file by scanning for known pitfalls, then use the renderer to catch render-time issues. Use it after editing a problem and before publishing, following the seven-step lint workflow.
7.2: Setting Up the PG Renderer
Walks through a first-run checklist, a reproducible edit loop, and a short a€œdonea€ checklist that matches biology authoring needs. After finishing, you can keep one file and seed stable, record what changed, and verify that the prompt and blanks read correctly before moving on. Use it during day-to-day authoring when you want fast iteration, clear notes, and predictable fixes before ADAPT preview.
7.3: Scripting and Automation of the PG Renderer
Leads with the lint_pg_via_renderer_api.py script that reads a local .pg file, posts it to the renderer, and reports errors and warnings in one step, with CLI options, batch linting, and full copyable source. After finishing, you can lint any PG file with a single command, batch-check a directory, and understand the renderer JSON response well enough to build custom tools. Use it as your first stop for command-line testing.
7.4: Common Mistakes and How to Fix Them
Adds a "start here" triage, a common-failures table, and a minimal debugging note template on top of the existing QA pass. After finishing, you can isolate render failures quickly, map symptoms to likely causes, and sweep variants without losing the reproducible case. Use it as a repeatable first-pass QA loop before an ADAPT preview and before importing or publishing a revised problem.
7.5: Testing Randomization and Edge Cases
Explains why randomization testing matters, how to sweep variants by changing seeds, five guard patterns (zero denominators, repeated values, invalid domains, sorted hash keys, shuffle-after-define), and boundary value testing for biology domains. After finishing, you can test multiple seeds systematically and add guard patterns to prevent impossible values. Use it when building any problem with randomization, and test at least five seeds before publishing.
7.6: QA Checklist Before Publishing
Provides a 12-row prevention checklist covering code-level issues (variable order, my keyword, backslash references, blocked HTML tags, required macros) and a 9-row author checklist covering student-facing issues (multiple variants, answer testing, edge cases, solution block). After finishing, you can confirm a problem is ready to publish with no silent failures. Use it as your final pre-release routine whenever you add or revise problems in a bank.

Search

Text Color

Text Size

Margin Size

Font Type