Step 1: Prompt layer

This is the highest-leverage near-term step because it creates a common language for all downstream workflows. The supplement describes a structured prompting strategy with an introductory sentence, a study-setting description, participant information, treatment content, and the outcome question (Hewitt et al. 2024b). That same structure can be reused for chapter simulations even when the design is not identical.

For the current package, this would let us move away from hard-coding prompts in user scripts and toward explicit prompt templates that are inspectable, versionable, and easier to document.

Step 2: Condition-based simulation wrapper

This is the step that would bring nalanda closest to the paper’s core design. The current one-turn workflow already has much of the required mechanics, but it is organized around books and chapters rather than experimental conditions. A dedicated wrapper should make conditions, control groups, and outcomes first-class objects. The resulting outputs can then be handed off to downstream tools for mean comparisons, contrasts, or other inferential analyses when needed.

Step 3: Descriptive summaries and calibration

This step should be narrower than a full contrast-analysis framework. The role of nalanda here is to produce package-native summaries that are useful for inspection, plotting, and workflow handoff. For pre/post designs, that includes metrics such as within-unit deltas. For post-only condition-labeled designs, that includes condition-level summaries and calibration helpers. The papers show strong rank-order prediction, but they also argue that absolute effect magnitudes are systematically overstated without calibration (Hewitt et al. 2024a, 2024b). That makes calibration worth supporting, even if inferential testing remains outside the package.

Step 4: Demographic profile infrastructure

This matters, but I do not think it should block the earlier steps. The subgroup analysis in the supplement suggests that matched demographic prompts gave only small or no predictive advantages for gender and ethnicity, with somewhat more benefit for party (Hewitt et al. 2024b). That makes demographic conditioning important, but not the first dependency.

Step 5: Ensemble controls

The supplement explicitly reports that predictive accuracy improved as the number of prompts in the ensemble increased (Hewitt et al. 2024b). For that reason, ensemble prompting should not remain an implicit user choice. It should be represented as a documented object or argument in the package API.

Step 6: Cumulative chapter designs

This step is especially relevant for the book project, even though it goes beyond the paper’s main experimental setup. The existing pre/post framework is a natural base for cumulative designs because it already keeps the logic of before/after change separate from prompt execution. The hard part is deciding how accumulated reading history should enter later prompts.

Transferable immediately

1. Prompt standardization. The package would benefit from prompt templates that separate intro text, study framing, identity or profile information, chapter text, and outcome questions.
2. Prompt ensembles. Rather than relying on one canonical chapter prompt, we could average over several introductory phrasings or framing variants.
3. Optional richer participant context. The current identity-based context could be extended to richer demographic profiles, especially in cases where subgroup interpretation matters.
4. Separation of raw and calibrated results. The package already tends to preserve raw outputs and compute metrics downstream. That is a good fit for calibration as well.

Transferable with design adaptation

1. Condition-ready chapter outputs. A chapter can be treated as a treatment condition and stored alongside a no-reading control, placebo chapter, or alternative chapter, with the resulting outputs passed downstream for comparison.
2. Megastudy-style ranking. Sets of chapters or chapter framings could be compared as candidate interventions, much as the papers compare many treatments within one study.
3. Cumulative exposure. Later chapters could be modeled as interventions delivered after earlier ones, with prior material either preserved in memory or compressed into an accumulated summary state.

Less transferable without stronger validation

1. The exact 0.56 calibration factor. This number was estimated for the paper’s primary archive of U.S. survey experiments and should not be assumed to transfer automatically to chapter-level reading interventions.
2. Claims about subgroup benefits from demographic matching. The paper’s subgroup findings are informative, but chapter interventions may produce different patterns of heterogeneity.

Why not pre-adjust inside simulation functions?

This would make the raw model output harder to inspect, harder to compare across calibration schemes, and harder to validate later. It would also blur the line between model execution and statistical post-processing.

Why not leave calibration entirely to user scripts?

That is flexible, but it is easy to do inconsistently. If calibration is part of the recommended workflow, the package should provide a standard path for it.

Recommended compromise

For metrics that nalanda already owns conceptually, summary functions should be able to compute both raw and optional adjusted outputs. For example, if a summary function computes delta_outgroup, a calibrated variant could appear as adjusted_delta_outgroup when a calibration factor is supplied.

This has several advantages:

1. raw outputs remain accessible,
2. calibration remains explicit,
3. multiple calibration schemes can coexist,
4. user scripts remain simpler and less error-prone.

For future condition-based workflows, a better boundary may be to provide a small helper that adjusts already-computed effect columns, regardless of where those effects were estimated. In other words, nalanda does not need to own contrast estimation in order to support calibration.

Such a helper could work on a user-supplied column and append:

1. adjusted_effect
2. calibration_factor
3. calibration_source

The package default should probably be calibration = NULL, with named presets available for known settings. A preset corresponding to the Hewitt et al. primary archive could reasonably use 0.56, but that should be framed as a setting-specific option rather than a universal package default.

prompt_bank

The prompt_bank object would formalize the reusable prompt pieces that are currently spread across ad hoc strings in scripts. Conceptually, it should be a named list or tibble-backed object with a small, inspectable schema.

At minimum, a prompt_bank should contain:

1. intro_variants: short opening instructions or framing sentences.
2. setting_template: the general study description, such as survey context or reading-task context.
3. profile_template: a template for participant description, including placeholders for identity or demographic fields.
4. stimulus_template: a wrapper describing how the treatment text or chapter text is introduced.
5. outcome_template: the question and response-scale wording.
6. scenario: a label such as "survey_experiment", "book_prepost", or "book_cumulative".
7. metadata: version, source paper, and notes.

In practice, one useful design would be:

prompt_bank <- list(
  intro_variants = c(
    "You will be asked to predict how people respond to various messages.",
    "Can reading a message affect people's attitudes and actions?"
  ),
  setting_template =
    "Social scientists often conduct research studies using online surveys. The text below is from one such survey conducted on a large, diverse population of research participants.",
  profile_template =
    "Participant X is a {ideology}, {age}, {ethnicity}, {gender} participant with {education}. Politically, Participant X identifies as '{party}'.",
  stimulus_template =
    "Please read the material below. {stimulus_text}",
  outcome_template =
    "{outcome_text} Please choose a number from {scale_low} to {scale_high}.",
  scenario = "survey_experiment",
  metadata = list(source = "hewitt_ashokkumar_2024")
)

For book workflows, a related prompt bank might swap in a chapter-specific setting_template while keeping the same overall structure.

ensemble_size

I am imagining ensemble_size as more than a bare integer, even if the user API initially accepts an integer. Internally, it would be useful to represent the ensemble settings as a small control object.

At minimum, this object should capture:

1. n: number of prompt variants to use per condition or chapter.
2. method: whether prompts are sampled randomly, cycled deterministically, or exhaustively enumerated.
3. replace: whether prompt variants may repeat.
4. weights: optional prompt weights if some variants are meant to count more.
5. pooling: whether outputs are averaged at the response level, condition mean level, or effect level.
6. seed: a seed strategy for reproducibility.

Conceptually:

ensemble_size <- list(
  n = 8L,
  method = "sample",
  replace = TRUE,
  weights = NULL,
  pooling = "effect",
  seed = 42L
)

For an early implementation, it would be enough to let users pass ensemble_size = 1, 4, or 8, while storing the richer object internally. That would keep the public API simple but leave room to expand.

demographic_profiles

The demographic_profiles object should represent either a fixed set of profiles or a sampling frame from which profiles are drawn. This is important because the papers do not only vary wording; they also vary the participant being simulated (Hewitt et al. 2024a).

At minimum, each profile should be able to store:

1. profile_id
2. gender
3. age
4. ethnicity
5. education
6. ideology
7. party
8. weight
9. label

Conceptually:

demographic_profiles <- tibble::tibble(
  profile_id = c("p1", "p2"),
  gender = c("Female", "Male"),
  age = c("30-39", "Over 60"),
  ethnicity = c("White", "Black"),
  education = c("College", "Some college"),
  ideology = c("Conservative", "Moderate"),
  party = c("Strong Republican", "Lean Democrat"),
  weight = c(0.5, 0.5),
  label = c("profile_1", "profile_2")
)

For nalanda, this object could support at least three modes:

1. Identity-only mode, close to the current package design.
2. Fixed-profile mode, for exact reproducibility with a specified profile set.
3. Weighted-sampling mode, for approximating a target population.

For chapter work, the immediate value may be greatest for party or ideology, with richer demographic fields becoming more important when studying subgroup heterogeneity or matching a target population.