Asana used GPT‑6 Astra in Codex to analyze and rework the browser agent inside its StackAI platform. In tests, the optimized GPT‑6.1 Sol workflow ran 76× cheaper and about 5× faster than the original production configuration.
Why they did it
StackAI enables customers to build no‑code workflows that navigate websites, fill forms and collect data. At Asana’s scale, small inefficiencies in those workflows compound into significant runtime and model costs. Frank Hidalgo, PhD, StackAI’s CTO, tasked GPT‑6 Astra in Codex with investigating the agent, proposing fixes and measuring the impact; a process that would have taken one to two months by hand was done in roughly a week.
Study design
The team ran a 144‑run study comparing GPT‑6.1 Sol and three other frontier models referenced as Models A, B and C. They tested two history budgets (120,000 and 480,000 characters) and six caching/screenshot policies, each configuration executed three times per model. The task for every configuration was identical: collect six fields for each of 32 books from a public demo catalog, representative of customer workloads.
Key findings and changes
GPT‑6 Astra mapped the codebase and found that the agent cached fixed instructions and tool definitions but did not cache the growing browsing history of page text and screenshots. As a result, every request resent full history content, increasing costs. The agent also frequently dropped older screenshots and trimmed text at many steps; those edits changed the history, so simply caching the history would not have solved the issue.
Hidalgo selected three fixes proposed by GPT‑6 Astra for testing:
- extend caching to include browsing history,
- increase the amount of text retained (larger history budget),
- remove screenshots in batches instead of at every step.
The best policy allowed screenshots to accumulate up to 20 before trimming back to the most recent one, preserving earlier history unchanged for longer intervals. Combined with the larger 480,000‑character history budget, this became the optimized workflow.
Measured impact: cost and runtime
Under the optimized workflow on GPT‑6.1 Sol, the mean estimated model cost was $0.47 per run and runtime about four minutes — representing a 76× cost reduction and roughly 5× speedup compared to the original production setup on Model B.
Selected comparisons from the study:
- On original Model B, estimated model cost was at least $36.21 per run (some original runs hit step limits before finishing), and optimization reduced that to $1.24 per run — a 29× reduction.
- The optimized GPT‑6.1 Sol ran at $0.47 per run, about 2.6× cheaper than the optimized Model B. On GPT‑6.1 Sol with the larger history budget and the new cache/screenshot policy, cost fell 4× from $1.97 to $0.47 per run.
- Cost per call improved because 89% of input tokens came from cache, which were priced at about 5% of the uncached input cost.
- Runtime fell from at least 22.5 minutes on the original Model B setup to roughly four minutes with the optimized GPT‑6.1 Sol workflow.
All runs under the optimized workflow completed the task and returned correct answers. Increasing the history budget also raised the rate of completed answers: with GPT‑6.1 Sol and the smaller history budget, only 3 of 18 runs returned an answer, whereas with the larger budget all 18 runs produced correct answers.
How changes were implemented
After quick experiments, the team refactored the code so one frontend and backend could support multiple parallel workflows with separate settings. GPT‑6 Astra executed the workflows, inspected requests, usage records and outputs, and separate model sessions reviewed the results. All requests, traces and results were recorded in Command, Asana’s software delivery platform; the findings were converted into tickets and pull requests, and the changes were deployed to production.
Asana has released the browser navigation changes in StackAI and is building tools to make repeat experiments easier. The team plans to integrate such testing into the platform’s evaluations so customers and internal teams can compare cost, runtime and answer quality when configuring agents.
GPT‑6 Astra in product testing
Asana now uses GPT‑6 Astra in Codex to test product features before release: Astra navigates the platform, tries different inputs and files bug reports for human QA reviewers. Hidalgo sees this as the basis for a new development lifecycle in which many cloud agent sessions test features in parallel.
More information
The complete study and detailed writeups are available on the Asana and StackAI blogs.
