surveyBurden estimates how much response effort a survey requires. It scores every question with the GfS+ point scheme (extending Heimgartner & Axhausen 2024). It can read a survey and follow its display logic to show how burden varies across the paths through the survey. If you have response data, the package can also tell you the burden per-respondent. For any platform, surveyBurden can give you the per-question GfS+ scores and a survey total, from a simple table of your questions (validate_catalogue() then score_burden()); a burden range per path if you also list which questions each path shows (path_burden()). The package works best with Qualtrics (either .qsf export in JSON or through the API), and can do all of the above, worked out automatically, plus the full report: routing and skip logic, burden across paths, respondent-level predictions, and analysis of response data and completion times. surveyBurden is an independent project with no affiliation with Qualtrics or any other survey platform, and no platform provides support for it.
| What you have | Start with | What you get | Guide |
|---|---|---|---|
| A survey on any platform, as a list of questions | score_burden(validate_catalogue(catalogue)) |
per-question scores and a total | vignette("extensions") |
A Qualtrics survey (.qsf or API) |
burden_report(qsf) |
a model-based structural burden profile across the survey's paths | vignette("reading-the-report") |
| A Qualtrics response export | respondent_burden(qsf, responses) |
respondent burden: the GfS+ points displayed to each respondent | vignette("respondent-burden") |
| A response export, to model break-off | respondent_burden(qsf, responses, by = "block") |
per respondent and block: routed in, and the burden displayed | vignette("respondent-burden") |
| A response export with completion times | validate_times(qsf, observed) |
a check of respondent burden against observed time, and a survey-specific rate | vignette("calibration") |
| A draft you want to make lighter | score it before and after an edit | where the burden is, why, and how much a change removes | vignette("survey-revision") |
surveyBurden is not on CRAN. Install the development version from GitHub:
# install.packages("pak")
pak::pak("pmpk20/surveyBurden")
# or, with remotes:
# install.packages("remotes")
remotes::install_github("pmpk20/surveyBurden")To work from a local clone, use pkgload::load_all("path/to/surveyBurden") for one-off use, or R CMD INSTALL path/to/surveyBurden to install it.
Dependencies: jsonlite, cli and tibble, plus httr2 only if you fetch from the Qualtrics API.
With a Qualtrics survey, one call gives the full report (this uses the demo survey shipped with the package). Not using Qualtrics? See Surveys from other platforms.
library(surveyBurden)
qsf <- system.file("extdata", "demo_travel_survey.qsf", package = "surveyBurden")
report <- burden_report(qsf)
report#> ── Survey Burden Report: Neighbourhood Travel Survey (demo) ────────────────────
#> Median burden: 123 GfS+ points (range 106-143 across 2 completing paths).
#> Expected minutes: ~10 (range 9-12).
#> Relative to benchmark: 0.3x the median of 399 points.
#> ...
The full printout adds the survey counts, the burden quantiles, burden by block, the heaviest questions, readability diagnostics, calculation certainty and QC warnings. vignette("reading-the-report") walks through every section.
summary(report) prints just the headline:
summary(report)
#> ── Neighbourhood Travel Survey (demo) ──────────────────────────────────────────
#> 26 questions, 12 pages, 12 blocks, 4 structural paths (2 complete).
#> Median burden: 123 GfS+ points (range 106-143 across 2 completing paths).
#> Expected minutes: ~10 (range 9-12).
#> Relative to benchmark: 0.3x the median of 399 points.report is a structured object, not printed text. Pull out each part:
report$survey # one-row tibble of the survey's structural counts
report$burden # min / p25 / median / p75 / max, in points, minutes and index
report$items # one row per question, with its GfS+ score and the rule used
report$paths # one row per structural path, with its burden floor and ceiling
report$blocks # burden totalled by block, in survey orderreport$survey--- questions, blocks, branch points, Loop & Merge blocks and structural paths.report$burden--- the headline spread of path burden across the survey.report$items--- every live question, its standard type, its point score, and ascore_flagsaying how the score was reached.report$paths--- each structural path, whether it completes or screens out, and the burden range along it.report$blocks--- question burden by block, with each block's share.
Scalar settings (the points-per-minute rate, the rare-value threshold) are stored as attributes: attr(report, "points_per_minute"), attr(report, "rare_threshold").
Each printed section is explained in vignette("reading-the-report").
The documentation uses these terms with fixed meanings. The key distinction: a path is a possibility in the survey's design; a respondent's path is the one a respondent actually took.
| term | meaning |
|---|---|
| path | one way through the survey that its flow allows: the blocks shown, given which branches are taken. Worked out from the survey design alone; no respondent is involved. |
| question burden | the GfS+ point score for one question, from its type and structure |
| path burden | the total question burden along one path. Display logic and loops can vary within a path, so a path has a spread of burdens rather than one number. |
| structural burden profile | the burden values a path's modelled display-logic states produce, with model weights (equal per state; each path's weights sum to 1). Not a respondent probability distribution. |
| respondent burden | the GfS+ points of every item displayed to one respondent, reconstructed from their responses: descriptive text and questions left blank included, each loop iteration counted (respondent_burden()). |
| calculation certainty | counts, for a given survey, of the parts of the calculation that are enumerated in full within the model and the parts that rest on documented approximations. A modelling diagnostic, not a check against the live survey. |
| readability diagnostics | counts of long question stems, long matrix labels, and long grids. Reading-load signals, reported separately, never folded into the GfS+ score. |
flowchart LR
any["Survey on any platform<br>(table of questions)"] --> val["validate_catalogue()"]
val --> sb["score_burden()"]
sb --> tot["per-question scores<br>+ survey total"]
src["Qualtrics survey<br>(.qsf file or API)"] --> read["read_qsf()"]
read --> score["parse_qsf() +<br>score_burden()"]
score --> paths["resolve_paths() +<br>path_burden_profile()"]
paths --> report["burden_report()"]
report --> out["verdict line +<br>8 report sections"]
Both ways in score questions with the same rules. For a Qualtrics survey, burden_report() runs the whole pipeline; the lower-level functions expose each step for inspection or reuse (see the reference table below).
surveyBurden scores every question with the GfS+ point scheme (extending Heimgartner & Axhausen 2024, Table 1), then follows the survey's flow and display logic to work out which questions can appear together and how burden varies across the structural paths the flow allows. The score for one question is its question burden; the total along one path is a path burden. The default result is a structural burden profile: each complete path carries equal weight, and within a path each modelled display-logic state does, so the reported "median" is the middle value of that model-weighted distribution, not the burden half of respondents exceed. Branch conditions are not checked against each other, so the extremes are not guaranteed to be reachable by a real respondent. Given response data, respondent_burden() gives each respondent's respondent burden, and the report adds its distribution over complete responses.
The package describes the survey, not the respondent: it does not claim that burden causes dropout, satisficing or slower responses. The methodology is set out in vignette("gfs-scoring") and vignette("paths-and-display-logic").
Set your credentials as environment variables, then pass a survey URL or ID to burden_report():
Sys.setenv(
QUALTRICS_API_KEY = "your-api-key",
QUALTRICS_BASE_URL = "https://yourdatacenter.qualtrics.com"
)
burden_report("https://yourorg.qualtrics.com/survey-builder/SV_xxxxxxxxxxxxxxxx/edit")Only the survey ID (SV_...) is needed, so a survey-builder link, a distribution link (.../jfe/form/SV_...), or the bare SV_ id all work. burden_report() fetches the survey definition, not its responses.
QUALTRICS_BASE_URL is the API datacenter host, which can differ from your survey-builder host. Check Account Settings, then Qualtrics IDs. Credentials are read from the environment and never appear in code.
You can only score a survey you can access. burden_report() reads the survey definition through the survey-definitions API, which requires a token with the "Manage Survey" permission on that survey --- normally a survey in your own Qualtrics account. A public participation link (.../jfe/form/SV_...) only lets a respondent take the survey; it does not expose the definition. To score someone else's survey you need either their .qsf export or an API token for the account that owns it.
The scoring rules do not depend on Qualtrics. Describe your questions in a data frame -- one row per question, with its type and, where relevant, its number of options, rows or columns -- and score it:
catalogue <- validate_catalogue(data.frame( question_id = c("Q1", "Q2", "Q3"), std_type = c("single_choice", "matrix", "open_text"), n_options = c(5L, NA, NA), n_rows = c(NA, 4L, NA), n_cols = c(NA, 5L, NA) )) scored <- score_burden(catalogue) sum(scored$gfs_points) # survey total, in GfS+ pointsvalidate_catalogue() checks the table and fills in sensible defaults for anything you leave out. The scores use exactly the same rules as for a Qualtrics survey. If your survey has routing, list which questions each path shows and path_burden() gives the burden range per path. vignette("extensions") has the full column list, a worked example, and what it would take for another platform's routing and skip logic to be read automatically, as Qualtrics' are.
burden_report() runs the whole pipeline. The lower-level functions expose each step.
| function | output |
|---|---|
read_qsf(source) |
internal survey object from a .qsf path, SV_ id or URL |
fetch_qsf(survey, api_key, base_url) |
the same object, fetched through the API |
parse_qsf(x) |
question catalogue, one row per live question |
classify_question(payload) |
standard question type for one question |
gfs_scheme() |
the GfS+ point scheme and the time conversion |
score_burden(catalogue, scheme) |
catalogue plus gfs_points, score_flag, score_basis |
resolve_flow(qsf, max_paths) |
the distinct block sequences (structural paths) |
resolve_paths(qsf, max_paths) |
per path: questions always shown, may be shown, never shown |
classify_reachability(catalogue, path_qids) |
reachability label for each question on a path |
path_burden(qsf, scheme, max_paths) |
burden floor and ceiling per path |
path_burden_profile(qsf, scheme, max_paths) |
model-weighted burden values per path, from the enumerated display-logic states |
summary_line(x, ...) |
a one-sentence burden summary |
calculation_certainty(qsf, scheme, max_paths) |
which parts of the calculation are enumerated in full within the model and which are approximated |
burden_report(x, scheme, profile, responses, max_paths, ...) |
the structured report object |
respondent_burden(qsf, responses, by) |
respondent burden after fieldwork, per respondent or per respondent and block |
exposure_person_period(x) |
person-period data for a discrete-time hazard model of break-off |
validate_catalogue(catalogue) |
check and coerce a hand-built question catalogue |
validate_times(qsf, observed, scheme, trim) |
respondent burden against completion times, and a fitted points-per-minute rate |
Every function has a help page: ?burden_report, ?score_burden, and so on. The guides cover the workflow end to end --- start with vignette("surveyBurden"), then vignette("reading-the-report"), vignette("survey-revision"), vignette("gfs-scoring"), vignette("paths-and-display-logic"), vignette("calibration"), vignette("respondent-burden") and vignette("extensions").
- The mapping from question types to GfS+ points is sometimes inferential; the package applies a documented rule and flags the question.
- The original GfS scheme predates modern web survey interfaces.
- Rendered text length cannot be recovered exactly from a
.qsf, sowords_per_lineis a configurable proxy. - Display-logic reconstruction depends on what the
.qsfencodes; unusual or externally controlled logic may need manual interpretation. - The structural burden profile is not a respondent probability distribution.
- The points-to-minutes conversion is a rule of thumb, not a completion-time forecast.
See vignette("calibration") for the detail.
Bug reports, questions and pull requests are welcome --- see .github/CONTRIBUTING.md. Participants are expected to follow the Code of Conduct.
citation("surveyBurden"), or see CITATION.cff.
MIT. Peter King, Institute for Transport Studies, University of Leeds.