# Blueprint for the eventual paper This is a writing plan and proposal text. It contains no experimental findings from the proposed method. The [initial manuscript draft](07-initial-manuscript.md) develops the Introduction and Method as connected academic prose. It uses the primary text-artifact setting and remains an untested proposal; this blueprint retains the plan for the eventual full paper. ## Recommended title **Cruise: Artifact-Guided Prompting for Fixed Agents Without External Evaluators in the Loop** Other defensible directions: - **From Delivered Artifacts to Better Follow-Up Prompts: A Study of Fixed-Agent Adaptation** - **When Are Artifact Critiques Enough? Prompt Adaptation Under Restricted Feedback** - **Preserving User Intent During Long-Horizon Prompt Adaptation** Choose the final title from the actual evidence. Use a title claiming better performance than GEPA only after a faithful direct comparison supports it. “Without external evaluators in the loop” describes adaptation, not the absence of independent assessment in the paper. ## Proposal abstract Language-model agents are often adapted through prompt optimization that uses execution traces, task scores, or reference-based feedback. In many deployed workflows, an external planner instead has access mainly to the artifacts an agent delivers. We propose to study artifact-guided prompting: a fixed planner inspects a delivered artifact against explicit user requirements and fixed principles, then generates a concrete instruction for the next turn of an unchanged worker. The adaptation loop receives no external performance scores, gold answers, or execution traces, and changes only follow-up prompts and bounded textual memory. Our proposed study separates within-task progress from transfer across tasks, and distinguishes artifact evidence from worker self-reports. We specify comparisons with generic continuation, self-refinement, reference-free prompt optimization, and scored optimization under explicit supervision and resource budgets. Independent assessment is withheld until outputs are sealed. Ablations test whether artifact access, fixed principles, and provenance-linked memory contribute beyond additional compute. The study is designed to identify both the usefulness and the limits of artifact-guided control, including regressions, unobservable errors, and drift from user intent. This abstract can accompany a research proposal. A completed-paper abstract must add measured results and revise its conclusions accordingly. ## Introduction: argument in five paragraphs 1. **Practical problem.** Users can state a destination but struggle to write a sequence of useful follow-ups. A fixed agent may make progress and then stop, repeat itself, or pursue increasingly narrow work. 2. **Existing solution space.** Describe GEPA, self-refinement, reference-free prompt optimization, and evolving memory accurately. Cite prior methods before naming the gap. 3. **Specific missing evidence.** Ask whether an artifact-only planner can choose useful next actions without runtime scoring or internal traces, under the same total compute as simpler alternatives. 4. **Method.** Introduce the immutable goal, fixed principles, restricted observation interface, generated next instruction, and bounded memory. Explain the boundary in one concrete example. 5. **Contributions.** State only what the completed work establishes: a formal regime, an implemented method, controlled evidence, and an analysis of failure modes. Do not list hoped-for wins as contributions already delivered. An appropriate sentence connecting to GEPA is: “We investigate a restricted-feedback setting in which the optimizer lacks the numeric selection signal and execution-level evidence used by GEPA's principal experiments.” This is a difference in setting to test, not a rebuttal of the validity of their results. ## Main-paper structure An approximately eight-page body is a useful drafting target, subject to the chosen venue's actual format. | Section | Approximate space | Required content | |---|---:|---| | 1. Introduction | 1 page | Problem, prior work, precise gap, scope of claims | | 2. Setting and related work | 1 page | Observation interface; mutable/frozen state; close alternatives; adaptation vs assessment | | 3. Method | 1.5 pages | Algorithm, principles, memory/provenance, budgets, termination | | 4. Experimental design | 1.5 pages | Tasks/splits, baselines, access matrix, independent scoring, statistics | | 5. Results | 1.5 pages | Primary effect with uncertainty, cost frontier, per-family outcomes, no cherry-picking | | 6. Mechanisms and failures | 1 page | Artifact interventions, compute control, memory drift, hidden-error cases | | 7. Limitations and conclusion | 0.5 page | Where the evidence applies and what remains unresolved | References and appendices carry exact prompts, configuration, full rubrics, additional runs, implementation details, and case studies. ## Figures to prepare after collecting evidence 1. **Method and information boundary.** Show goal + principles + artifact + memory entering the planner; next prompt entering the fixed worker; delivered artifact returning. Place the independent assessor outside the loop. Use crossed-out arrows for prohibited score/trace/source inputs. 2. **Quality against total spend.** Mean quality with task-clustered intervals at prespecified cost caps. Include all planning and discarded-candidate costs. 3. **Artifact intervention.** Show a controlled visible defect and the resulting follow-up behavior, alongside the measured effect across tasks. An example alone is illustrative, not causal proof. 4. **Memory drift and regressions.** Plot unsupported requirements and broken requirements across longer runs. Preserve unfavorable examples. Do not draw quantitative charts with invented data while drafting. Use labeled layout placeholders until results exist. ## Tables to prepare - **Table 1: access and supervision.** Each method's artifact, labels, score, trace, source, and memory access. - **Table 2: primary quality.** Per-family and overall quality, confidence intervals, repetitions, incomplete-run rate, and total cost. - **Table 3: mechanism ablations.** No artifact, stale artifact, no principles, no memory, and provenance-linked memory under matched budgets. - **Table 4: native optimizer comparison.** GEPA's development cost, test cost, amortization assumptions, and final quality; separate this from equal-access results. - **Appendix tables:** model/settings, task split sizes, rater agreement, all prespecified comparisons, and intervention accounting. ## Results-writing templates Use these only after replacing the brackets with verified outputs from the registered analysis: > At the prespecified budget, artifact-guided planning changed the mean requirement-satisfaction score by [difference] relative to generic continuation (95% task-clustered CI [lower, upper]; [number] tasks and [repetitions] whole-run repetitions). Its difference from ordinary self-refinement was [difference and CI]. The largest gains occurred in [family]; it regressed in [family or conditions]. > Under the native-supervision comparison, [method] achieved [quality] at [adaptation cost]. GEPA achieved [quality] with [development cost] plus [test cost]. These methods received different adaptation signals; the result measures the stated quality/cost tradeoff rather than isolating algorithm quality under identical supervision. If a contrast is uncertain, write that directly. If only one family improves, scope the conclusion to it. If labels are removed but a pairwise LLM judge remains, describe the method as reference-free, not judge-free. ## Anticipate reviewer questions with experiments | Likely question | Evidence needed | |---|---| | Is this just Self-Refine? | Faithful same-budget baseline and a measurable contribution from next-action principles or memory | | Is this already SPO, ProRefine, or ACE? | Method-level differences in observation, persistent task state, update rules, and selection; no novelty claims based only on names | | Why compare to GEPA when the setting differs? | Separate equal-access and native-supervision panels and define the scientific question of each | | Did you win because you made more calls? | Full usage accounting and worker-compute controls | | Is the planner secretly an evaluator? | Admit its qualitative judgment; show absence of an external acceptance oracle and define what that removes | | Did you change the harness to help one arm? | Frozen versions, configuration hashes, and the same worker stack across arms | | Is a better-looking output actually correct? | Independent factual/numerical/functionality checks, not just model preference | | Does memory help or distort the goal? | Provenance ablation and an audit of unsupported requirements | | Is this just the Launchboard demonstration? | New held-out tasks, complete repetitions, and honest labeling of the historical assisted run | | Can I reproduce it? | Complete allowed-input packets, exact submitted prompts, final artifacts, and analysis | ## How to turn the work into a paper without academic experience Start from the experiment record, not from academic-sounding prose. For every intended claim, keep a row containing the comparison, metric, task set, uncertainty, cost definition, and artifact/log that supports it. Draft Methods and Experimental Design before Results and Introduction. Then write the introduction around the contribution the evidence actually supports. Have a researcher familiar with prompt optimization review the comparison design and a statistically experienced reviewer inspect the analysis before test execution if possible. This is useful collaboration, not a prerequisite for beginning the pilot. Credit collaborators according to the venue's authorship rules and their actual contributions. Choose a venue after the pilot establishes scope. Check its current page limits, anonymization, artifact policy, and deadlines then; this plan makes no assumptions about an open submission window. A workshop or an empirical-study paper may fit a carefully bounded first result better than a sweeping RL-superiority claim. ## Readiness gate Before submission, ensure that every numerical statement is regenerated from recorded data, every baseline is faithful or labeled as an adaptation, all human steering is disclosed, all claimed strict runs satisfy the observation restrictions, and limitations match the negative cases. A clear, narrow supported claim is the intended outcome.