Thrive Holdings 与 OpenAI 如何通过融合从业者专业知识与 Codex 驱动的循环，为克里特岛会计师共同开发 Tax AI
真实系统在生产环境中的表现与实验室中不同，其故障方式在部署前难以预测。团队通常在上线后发现这些故障，然后花费数周检查边缘案例、调整提示词，并将生产反馈转化为持久的产品改进。这个反馈循环是手动且缓慢的，只有工程师推动时才能改进。但如今，借助精心设计的评估基础设施、直接接触从业者和真实环境的能力，以及 Codex 的前沿智能体能力，你可以构建能够自我改进的智能体。
在这篇文章中，我们将解析如何使用 Codex 构建这种类型的智能体。过去六个月，OpenAI 的前向部署工程师和研究人员与 Thrive Holdings 的工程师合作，为 Crete（在新窗口中打开） 的 30 多家会计师事务所网络共同开发了 Tax AI，以帮助准备日益复杂的纳税申报表。Tax AI 不依赖工程师发现和修复每个故障，而是利用 Codex 将生产使用转化为结构化信号，从而驱动自主改进。
Crete 的从业者每个季节准备数万份纳税申报表，这需要处理数百万份底层文档。对于中高复杂度的申报，仅数据输入每份申报就可能需要八小时，通常涉及杂乱的数据源、往年文档以及手动提取和计算。他们向我们指出，在报税季最繁忙的时期，税务准备是一个重大瓶颈。
为了解决这个问题，Tax AI 在本报税季处理了参与试点的 Crete 公司的 7,000 份纳税申报表。该系统自动化了准备 1040 和 1041 纳税申报表中许多耗时的流程，但比效率提升更引人注目的是，该系统本身比三个月前首次部署的版本有了显著改进。
可衡量的自我改进
在 Tax AI 中，从业者上传源文件以及任何客户特定的备注。然后，Tax AI 创建一份税务引擎提交，供审核。它为从业者节省了大约三分之一的税务准备时间，起草的申报表准确率高达 97%，并将吞吐量提高了约 50%，从而为他们留出更多时间与客户相处。
我们可以通过了解 Tax AI 在不需后续更正的情况下完成申报的准确率来量化这种改进。我们通过检查达到 75%、90% 或 100% 正确字段完成率的申报比例来衡量准确性。上线时，只有四分之一的申报达到了 75% 的正确字段完成率，但六周内，86% 达到了这一标准。在 90% 和 100% 正确字段完成率水平上，系统显示出更快的增长。这些阈值让我们能够实际了解不同申报仍需多少从业者跟进。
早期，Tax AI 处理较简单的工作，如 W-2 和 1099 表格。随着季节推进，它进入了更复杂的申报，涉及 K-1、附表以及更难的边缘案例。每项新能力比前一项节省了更多每份申报的时间，因为它承担的任务更困难且手动操作更耗时。我们至今仍看到持续的进展。
接下来，我们将介绍我们的团队如何共同设计 Tax AI，使其通过依赖三个关键支柱实现自我改进：1）专家从业者反馈，2）生产轨迹（从输入到最终输出的结构化历史记录），以及 3）基于定制评估的 Codex 驱动迭代循环，以实现持续、更快的产品开发。我们希望我们的经验对其他构建者有用，尤其是在从业者专业知识对塑造整体系统质量和其中运行的数据至关重要的领域。
随着 Tax AI 扩展到更复杂的申报，达到 75%、90% 和完全完成率的评分申报比例在报税季持续上升。
问题
当我们进入税务准备的更困难部分（K-1、租赁房地产附表以及需要跨多个源文件对账数值的税务表格）时，很明显，真正的挑战在于产品是否能让复杂的生产故障变得可见、可理解且可操作。
在产品早期，大部分更正都是手动的。从业者可以纠正系统错误，但产品并未捕获完整的上下文：申报前更改的数值可能反映了真正的提取遗漏、映射问题、缺失的产品支持或预期的工作流噪声。分类这些情况仍需工程团队跟进。工程师可以使用编码智能体，但系统尚未设计为在改进循环中有意义地使用 AI。我们没有信号来确定要攀登的正确山峰。
我们的方法：三部分循环
这促使我们围绕三个支柱设计系统：
- 贴近从业者： 从事工作的人需要引导产品学习的内容。他们的直觉和理解揭示了哪些错误重要，并帮助确定工作流的哪些部分值得下一步关注。
- 构建产品，使生产产生证据： 产品必须捕获的不仅仅是输入和输出；它需要捕获从源材料、提取字段和出处到下游提交和专家更正的完整路径。
- 创建 Codex 驱动的改进循环： 一旦生产问题变得可见且结构化，它们就可以成为发现、定制评估和范围明确的工程任务。然后，Codex 可以帮助调查、提出更改、针对目标和回归评估进行验证，并比纯手动迭代周期更快地推动产品前进。
下面的租赁房地产示例展示了该循环在实践中如何运作，引导您了解从业者更正如何成为结构化发现，然后成为评估目标，最后成为 Codex 范围的工程任务。
租赁房地产示例
租赁房地产收入在个人纳税申报表的附表 E 中报告。从工程角度来看，提取它的任务描述起来简单，但做好却很难。系统必须读取杂乱的源材料（手写备注、电子邮件、电子表格和其他客户文件），提取系统可以自信地映射到税务引擎的租赁房地产字段，并保留足够的证据，以便从业者批准或更正结果。下面的简化示例展示了这些源文件和提取输出可能的样子。
租赁房地产源包被规范化为带引用的字段，然后映射到下游税务引擎概念。
1. 从业者更正揭示了一个故障
代理预测值与税务申报表中的实际值之间的差异，可能反映了真实的提取遗漏，但也可能是从业者的偏好、税务引擎中从上一纳税年度申报表结转的值，或者在申报工作流程中其他地方引入或更改的值。从业者帮助我们识别这些情况，以便我们确定哪些操作需要从业者纠正或阻止提交。
由于我们能够详细查看这些纠正，我们将审查过程从一个终端性的、失败后的步骤转变为一个持续的学习循环。我们设计了工作流程，将专家操作作为结构化数据捕获。现在，每一次干预都会通过记录 Tax AI 提出的内容、从业者修改的内容以及最终进入申报表的内容，来反馈到产品的改进循环中。
2. 产品痕迹将纠正转化为评估
对于像租赁房产这样复杂的工作流程，系统必须保留从源文件到最终申报表之间发生的一切。在这条路径上，文档被组织、拆分和分类；租赁房产字段被提取并附上源材料的引用；这些值被映射到税务引擎中；从业者可能在申报前仍然会纠正它们。这些产品级别的痕迹使得调查失败发生的位置成为可能。为了将从业者的纠正转化为有用的评估目标，系统通过三个步骤处理它们：
- 捕获差异： 将 Tax AI 的输出与申报表进行比较，生成字段级别的审查行，捕获预期值、预测值以及差异是否看起来可操作。
- 分组相关失败： 将相似的审查行分组，以将重复的产品失败与预期的工作流程噪音区分开来。例如，重复的从业者纠正可能表明 Tax AI 经常遗漏“公平租赁天数”字段、错误处理“其他费用”，或混淆同一源包中的多个租赁房产。
- 将重复模式转化为评估目标： 一旦经过审查和衡量，重复的发现就会成为 Codex 改进的明确评估目标。
租赁房产审查行将重复的产品失败与预期的噪音分开，然后将可操作的案例转化为评估目标，为 Codex 提供一个可攀登的“山丘”。
3. 发现成为 Codex 可攀登的山丘
第三个支柱是创建一个能够根据这些新评估采取行动的工程循环。这正是 Codex 发挥核心作用的地方。
假设我们的评估管道标记出 Tax AI 始终遗漏“公平租赁天数”字段，而从业者则可靠地填写它。由于这个发现已经被打包成一个有针对性的评估集，包含代表性的源包和预期输出，Codex 可以直接在产品框架内调查根本原因。
Codex 不仅仅是在处理一个次优的最终输出。它同时检查痕迹、评估、代码库和技能：
- 调查管道： 检查源包、提取模式、映射器行为和代码路径，以确定问题是未支持的字段、遗漏的提取模式、源选择问题、映射器缺口还是评分器问题。
- 实施有针对性的修复： 扩展提取模式、改进租赁房产文档的源选择、更新税务引擎映射器，或者如果预期的工作流程噪音被计为失败，则优化评分器。
- 验证并提出： 重新运行有针对性的评估，运行更广泛的回归测试套件，并提交一个候选拉取请求供工程审查。
- 闭环： 将重复的从业者纠正转化为可衡量的工程任务。如果证据不明确或无法安全自动化，则该案例会路由回产品团队，而不是强制通过循环。
端到端的自我改进循环：生产痕迹揭示重复的字段级别纠正，这些纠正成为失败信号，Codex 可以结合痕迹、评估、代码库和技能进行检查。可操作的模式成为有边界的评估和候选产品变更；模糊的案例路由回工程师进行审查。每个已交付的改进都会为下一个循环创造新的生产证据。
如何使用 Codex 构建这个循环
租赁房产的例子代表了一种更广泛的可重用模式：利用生产工件和痕迹来改进代理的能力。给定来自生产数据的经过审查的发现、源痕迹、预期的税务引擎输出、相关代码示例和评估命令作为一组输入，Codex 可以在数周和数月内显著提高性能和准确性。这建立在我们关于驾驭工程和Symphony的工作中描述的原则之上，这些工作详细介绍了如何使任务对 Codex 可读、提供范围化的上下文和工具，以及将验证和人工审查作为环境的一部分。
这些证据不会自动成为 Codex 的任务。从业者的纠正可能反映了提取遗漏、映射问题、不支持的产品行为、税务判断或预期的工作流程噪音。只有在重复的差异经过审查并分组为可操作的发现后，系统才会将它们转化为具有明确成功条件的有边界任务。
我们将这种自动化应用于产品的一个有边界层。这一层执行提取并将源文档映射到税务工作流程中。工程师仍然负责架构、产品决策和发布。从业者通过他们已经在做的工作来引导改进循环：纠正提取的值、审查申报表和批准最终申报。
对于 Codex 来说，结果不是一个模糊的警报，而是一个范围化的工程任务，包含证据、可编辑的产品表面和明确的验证门控。一个代表性租赁房产任务的上下文可以总结如下：
一个有边界的 Codex 任务环境将可写的工作树 [1] 与只读的生产上下文 [5] 分开。工作树包含 Codex 可以检查或修改的范围化产品表面 [2]、定义成功的有针对性的评估和回归评估 [3]，以及编码了如何运行任务和尊重先前决策的可重用技能/文档 [4]。只读上下文提供生产痕迹、源文档、Tax AI 预测、最终申报表和税务引擎字段文档，以便 Codex 可以在不改变基础证据的情况下调查失败。
扩展到新领域
同样的循环也适用于租赁房产之外。租赁房产大约花了六周时间和大量的工程监督才达到 90% 的精确率和召回率，但这项工作产生了可重用的抽象、审查工件、评估约定和实现模式，使得支持类似复杂度的附表（如附表 C 和附表 A）变得更加容易。
税务AI证明了构建自我改进型智能体的可行路径。从业者通过提供服务生成高价值反馈信号，产品工作流将这些信号保留为结构化证据。基于评估的工程系统在改进措施进入生产环境前进行验证，而智能体驱动的循环机制则使系统持续处于自我优化的流动状态。
Thrive Holdings的组织架构使我们能够在特定行业复制这种环境。作为兼具所有者与运营者双重身份的控股公司，我们的联合工程团队能够直接与从业者及来自内部企业（如Crete）的生产数据协作——不是以供应商身份，而是作为合作伙伴。这意味着技术、产品与服务同属一个体系，帮助我们更快推进并打造卓越产品。
一位去年花费180小时处理税务申报的高级会计师，今年仅用了15小时。她将节省的部分时间用于逐一联系客户并指导他们完成申报——这种高触达服务在一年前还无法实现。其余时间则用于拓展新客户和开发新服务项目。
如今，我们的团队正将税务AI的三部分设计蓝图应用于Thrive Holdings旗下其他领域的工作流构建：包括簿记与审计等会计工作流，以及IT服务台自动化等运营工作流。跨领域、跨行业的自我改进型智能体正展现出更广阔的前景。最优秀的智能体始终由人类引导学习，随着时间推移变得更具能力、更值得信赖、也更有价值。
欲了解更多参与此项目的OpenAI团队信息，请联系我们。
How Thrive Holdings and OpenAI co-developed Tax AI for Crete accountants by fusing practitioner expertise with a Codex-driven loop
Real-world systems behave differently in production than they do in a lab, breaking in ways that are hard to anticipate before deployment. Teams often discover those failures after launch, then spend weeks inspecting edge cases, adjusting prompts, and translating production feedback into durable product improvements. The feedback loop is manual and slow, and only improves when an engineer advances it. But today, with thoughtfully designed eval infrastructure, direct access to practitioners and real world environments, and the frontier agentic capabilities of Codex, you can build agents that self-improve.
In this post, we’ll unpack how we used Codex to build this type of agent. Over the past six months, OpenAI forward deployed engineers and researchers along with Thrive Holdings’ engineers collaborated to build Tax AI alongside and for Crete(opens in a new window)’s network of 30+ accounting firms to help prepare increasingly complex tax returns. Instead of relying on engineers to find and fix each failure, Tax AI uses Codex to turn production use into structured signals that fuel autonomous improvement.
Crete practitioners prepare tens of thousands of tax returns each season which requires working through millions of underlying documents. For medium- to large-complexity filings, data entry alone can take eight hours per return, often involving messy data sources, prior-year documents, and manual extraction and calculation. They pointed us to tax preparation as a significant bottleneck during the busiest stretch of tax season.
To solve this problem, Tax AI processed 7,000 tax returns across the Crete firms that participated in the pilot this tax season. The system automates much of the time-intensive process of preparing 1040 and 1041 tax returns, but even more compelling than the efficiency gains is that the system itself is measurably better than the version that was first deployed three months ago.
Measurable self-improvement
In Tax AI, practitioners upload source files along with any client-specific notes. Tax AI then creates a tax engine submission, ready for review. It saves practitioners about a third of their time on tax preparation, drafts returns with up to 97% accuracy, and increases throughput by about 50%, creating more room for them to spend time with clients.
We can quantify this improvement by understanding how accurately Tax AI can complete a return without needing correction later. We measure accuracy by checking what share of returns reach 75%, 90%, or 100% correct field completion. At launch, only a quarter of returns were at 75% correct field completion, but within six weeks, 86% hit that mark. The system showed even faster growth at the 90% and 100% correct field completion levels. These thresholds give us a practical view of how much practitioner follow-up different returns still require.
Early on, Tax AI handled simpler work, like W-2s and 1099s. As the season went on, it moved into more complex returns with K-1s, schedules, and harder edge cases. Each new capability saved more time per return than the last because the tasks it took on were harder and more time consuming to do manually. We continue to see ongoing progress today.
Next, we’ll walk through how our teams co-engineered Tax AI to be self-improving by leaning on three critical pillars: 1) expert practitioner feedback, 2) production traces (a structured history from inputs through final output), and 3) a Codex-driven iteration loop based on tailored evals to enable continuous, faster product development. We hope our experience will be useful to other builders in domains where practitioner expertise is key to shaping the quality of the overarching system and the data running through it.
As Tax AI expanded into more complex filings, the share of scored returns reaching 75%, 90%, and full completion continued to rise through tax season.
The problem
As we pushed into harder parts of tax preparation (K-1s, rental real estate schedules, and tax forms where values had to be reconciled across multiple source files), it became obvious that the real challenge was whether the product could make complex production failures visible, understandable, and actionable.
In the early days of the product, most of the correction was manual. Practitioners could correct system errors, but the product did not capture the full context: a changed value before filing might reflect a true extraction miss, a mapping problem, missing product support, or expected workflow noise. Sorting those cases out still required follow-up from the engineering team. Engineers could use coding agents, but the system was not yet designed to use AI meaningfully inside an improvement loop. We did not have the signal to identify the right hill to climb.
Our approach: a three-part loop
That led us to design the system around three pillars:
- **Stay close to practitioners:**The people doing the work need to steer what the product learns. Their intuition and understanding reveal which errors matter and help inform which parts of the workflow are worth focusing on next.
- **Build the product so production creates evidence:**The product has to capture more than just inputs and outputs; it needs to capture the full path from source material, to extracted fields and provenance, to downstream submission and expert correction.
- **Create a Codex-driven improvement loop:**Once production issues are visible and structured, they can become findings, tailored evals, and scoped engineering tasks. Codex can then help investigate, propose changes, validate them against targeted and regression evals, and move the product forward faster than a purely manual iteration cycle.
The rental properties example below shows how that loop works in practice, walking you through how a practitioner correction becomes a structured finding, then an eval target, and finally a Codex-scoped engineering task.
Rental property example
Rental property income is reported on Schedule E of an individual tax return. From an engineering perspective, the task of extracting it is simple to describe but hard to do well. The system has to read messy source material (handwritten notes, emails, spreadsheets, and other client files), extract the rental-property fields the system can confidently map to the tax engine, and preserve enough evidence that a practitioner can approve or correct the result. The simplified example below shows what those source files and extracted outputs might look like.
A rental property source package is normalized into cited fields before those are mapped to downstream tax engine concepts.
1. A practitioner correction reveals a failure
A difference between the agent-predicted value and the actual value from the filed tax return might reflect a true extraction miss, but it could also be a practitioner preference, a value carried forward from a prior-year return in the tax engine, or a value introduced or changed elsewhere in the filing workflow. Practitioners helped us discern those cases so we could identify which actions required a practitioner correction or blocked a submission.
Because we could see these corrections in detail, we transformed the review process from a terminal, post-failure step into a continuous learning cycle. We designed the workflow to capture expert actions as structured data. Now, every intervention feeds the product's improvement loop by recording exactly what Tax AI proposed, what the practitioner modified, and what ultimately went into the filed return.
2. Product traces turn corrections into evals
For a complex workflow like rental properties, the system has to preserve what happens between the source files and the filed return. Along that path, documents are organized, split, and classified; rental-property fields are extracted with citations back to the source material; those values are mapped into the tax engine; and practitioners may still correct them before filing. Those product-level traces make it possible to investigate where a failure occurred. To turn practitioner corrections into useful evaluation targets, the system processes them in three steps:
- Capture the difference: Tax AI’s output is compared with the filed return to produce field-level review rows that capture the expected value, predicted value, and whether the difference appears actionable.
- Group related failures: Similar review rows are grouped to separate recurring product failures from expected workflow noise. For example, repeated practitioner corrections might show that Tax AI often misses fair-rental-day fields, mishandles “other expenses,” or confuses multiple rental properties across the same source package.
- Turn repeated patterns into eval targets: Once reviewed and measured, repeated findings become clear eval targets for Codex to improve.
Rental property review rows separate recurring product failures from expected noise, then turn the actionable cases into evaluation targets that give Codex a hill to climb.
3. The finding becomes a hill to climb for Codex
The third pillar is creating an engineering loop capable of acting on these new evals. This is where Codex becomes central.
Suppose our eval pipeline flags that Tax AI consistently misses the "fair rental days" field, while practitioners reliably fill it in. Because this finding has already been packaged into a targeted eval set, with representative source packages and expected outputs, Codex can investigate the root cause directly within the product scaffold.
Codex isn’t working solely with a sub-par final output. It inspects the trace, eval, repo, and skills together:
- Investigate the pipeline: Inspect source packages, extraction schemas, mapper behavior, and code paths to determine whether the issue is an unsupported field, a missed extraction pattern, a source-selection problem, a mapper gap, or a grader issue.
- Implement targeted fixes: Extend the extraction schema, improve source selection for rental-property documents, update the tax-engine mapper, or refine the grader if expected workflow noise is being counted as a failure.
- Validate and propose: Rerun the targeted eval, run broader regression suites, and surface a candidate pull request for engineering review.
- **Close the loop:**Turn a recurring practitioner correction into a measurable engineering task. If the evidence is ambiguous or not safely automatable, the case routes back to the product team instead of being forced through the loop.
The end-to-end self-improvement loop: production traces surface repeated field-level corrections, which become failure signals that Codex can inspect alongside the trace, evals, repo, and skills. Actionable patterns become bounded evals and candidate product changes; ambiguous cases route back to engineers for review. Each shipped improvement creates new production evidence for the next cycle.
How to use Codex to build this loop
The rental property example is emblematic of a broader reusable pattern: using production artifacts and traces to improve an agent’s capabilities. Given reviewed findings from production data, source traces, expected tax-engine output, relevant code examples, and eval commands as a set of inputs, Codex can materially improve on performance and accuracy over weeks and months. This builds on the principles described in our work onharness engineering and Symphony, which walk-through how to make tasks legible to Codex, provide scoped context and tools, and keep validation and human review part of the environment.
That evidence does not become a Codex task automatically. A practitioner correction may reflect an extraction miss, a mapping issue, unsupported product behavior, tax judgment, or expected workflow noise. Only after repeated differences have been reviewed and grouped into an actionable finding does the system turn them into a bounded task with a clear success condition.
We apply this automation to a bounded layer of the product. This layer performs extraction and maps source documents into tax workflows. Engineers remain responsible for architecture, product decisions, and shipping. Practitioners steer the improvement loop through the work they already do: correcting extracted values, reviewing returns, and approving final filings.
For Codex, the result is not a vague alert but a scoped engineering task with evidence, editable product surfaces, and explicit validation gates. The context for a representative rental property task can be summarized as follows:
A bounded Codex task environment separates the writable worktree [1] from read-only production context [5]. The worktree contains the scoped product surface Codex can inspect or modify [2], the targeted and regression evals that define success [3], and reusable skills/docs that encode how to run the task and respect prior decisions [4]. The read-only context provides the production trace, source documents, Tax AI prediction, finalized return, and tax-engine field documentation, so Codex can investigate the failure without mutating the underlying evidence.
Expanding to new domains
The same loop applies beyond rental properties. Rental properties took about six weeks and substantial engineering oversight to reach 90% precision and recall, but that work produced reusable abstractions, review artifacts, eval conventions, and implementation patterns that made it easier to support similarly complex schedules such as Schedule C and Schedule A.
Tax AI proves a path to building self-improving agents. Practitioners generate high-value feedback signals by delivering the service. Product workflows preserve those signals as structured evidence. Eval-backed engineering systems validate improvements before they reach production, and an agent-powered loop keeps the system in a continuous self-improving flow.
Thrive Holdings’ structure allows us to replicate this environment in specific industries. Holdings is both an owner and operator, so our combined engineering teams are able to work directly with practitioners and production data from inside businesses like Crete, not as a vendor but as partners. This means the technology, the product, and the service all sit under one roof to help us move faster and build exceptional products.
One senior accountant who spent 180 hours on tax prep last year spent only 15 hours on it this year. She put that time in part toward calling every one of her clients and walking them through their returns, a level of high touch service that wasn’t possible a year ago. The rest of that time she used to take on new clients and expand to new service offerings.
Together, our teams are now using the same three-part design from Tax AI as a blueprint for building workflows in other domains across Thrive Holdings(opens in a new window); accounting workflows such as bookkeeping and audit, and operational workflows such as IT help desk automation. Across domains and industries, the broader promise of self-improving agents holds. The best agents are steered by people to learn to become more capable, more trusted, and more valuable over time.
To learn more about the OpenAI team that worked on this project,get in touch.
本文内容采集自官方网站，排版和翻译可能与原页面存在差异。
阅读官方全文