Replacing the research pile with a briefing agent

District 3 is a publicly funded startup incubator at Concordia University in Montreal, processing roughly 200 to 250 applications per month. When an application lands, an operations team member spends three to four hours on background research: scouring LinkedIn, checking founder websites, mapping the competitive landscape, looking for patents and press. They write hasty bullet-point notes and assign a program stream. Stream leads walk into panel meetings and spend their first ten minutes just orienting themselves.

The research is the bottleneck. It's tedious and inconsistent. Two people reviewing the same application can produce dramatically different notes. The quality of the research dictates the quality of the panel's decision, yet it's treated as rote admin work.

mapping the organization before designing for itthe pain points that justified building something newfive groups, each with different needs and fears. The system has to serve all of thempicking which problems AI can solve without crossing into human territoryevery manual step, every handoff, every place it stalls

I built an AI system that eliminates the research bottleneck. When a new application is submitted, an agent autonomously generates a research brief: founder profiles with verified sources, competitive analysis, SDG alignment assessment, stream classification with reasoning, a scored evaluation rubric, risk flags, and interview questions for both operations staff and panelists, all cited to real URLs or application fields.

The human can now open an application that's already deeply researched. Operations shifts from doing research to reviewing research. Stream leads arrive at panel meetings with context instead of spending their first ten minutes catching up.

Every brief has nine sections, and every claim in them cites a URL, an application field or a knowledge base document.

Version 1 of this system was a scripted pipeline, a chain of prompts with predetermined steps and retry logic. It broke constantly. If a LinkedIn page was down or a founder didn't have a website, the whole chain derailed. Wrong output was worse than no output, because it created more work for the ops team to correct.

For version 2, I replaced choreography with agentic design. Instead of a task list, the agent gets an environment: knowledge to reference, tools to use, a goal to achieve, and guidelines for quality. Within that environment, the agent decides what to research, in what order, and how deep to go.

knowledge, tools, guidelines, goal. The agent decides the rest

the full research loop, with web research cascade, observer lane, and post-completion reviewer

The agent operates in a loop of research, assessment, and output. It loads D3's knowledge base, reads the application, creates a research plan, then executes that plan phase by phase. After each phase, it self-assesses its confidence and decides whether to proceed, retry with different sources, or flag for human review.

Brief sections are emitted incrementally as the agent completes them, not batched at the end. This means the brief builds in real time, and observers can watch the research unfold through a live server-sent event stream.

The key design insight is the backtracking loop. After each self-assessment, the agent rereads everything it has already published and asks: "Does anything I've already published need updating in light of what I just learned?"

That loop is how the system produces an honest, non-linear brief rather than a first-pass report.

The useful artifact is the audit trail: what the agent tried, what failed, when it asked for help, and the brief it produced.

This is the literal run log: fetch failures, escalations, self-assessments, emitted sections, and final review flags.

the same run, turned into something a reviewer can use

Nine cited sections, ranked flags, and a cleaner starting point for the ops team and panelists.

To research a URL, the agent works down five strategies until one returns something.

Even the research tool follows the sandbox pattern: the tool provides information, page content plus navigation links, and the agent provides judgment about which links are worth following.

The system has two distinct moments where humans interact with the agent's work, each designed for a different purpose.

The AI handles all research, analysis, cross-referencing, and structured output. It is explicitly not responsible for any decision or communication. It cannot accept or reject an applicant, send an email, or route a startup to a stream lead.

The tools for these actions don't exist, so no amount of clever prompting can unlock them. The boundary is architectural rather than policy.

the tools for decisions and outreach don't exist

The decision that stays human is accept or reject after the pitch meeting. Institutional judgment, founder rapport, and cohort composition mix there in ways an AI can't replicate. The brief hands the panel a researched starting point; the judgment call stays with them.

Because the agent has tools, knowledge, and the freedom to reason about what it finds, it catches things that no scripted system would. The insider connection above was one example; there is no "check if the applicant works at D3" step in the system. Another from the same test run:

These are the kinds of findings that justify the agentic approach. A scripted pipeline would check predetermined sources in a predetermined order. The sandbox lets the agent follow leads, cross-reference across sources, and surface patterns that emerge from the data.

every annotation is something that broke in productionthe research wall that shaped every V2 decision

Five lessons from V1 shaped V2.

Trust calibration. The brief is good enough that ops might rubber-stamp instead of reviewing. The self-assessment loops and human review flags resist this, and the system tells you when it's uncertain. The stronger safeguard is organizational: keeping humans accountable for the decisions the system refuses to make.

Other scaling concerns include API cost management at 200+ briefs per month, rate-limiting from external sources like LinkedIn, and the need for a proper database as the file-based storage approach won't hold under concurrent load. The trust calibration problem interests me most because no code change fixes it. The system makes human oversight easier; it can't force anyone to exercise it.

I had been circling this idea since 2022, when it became obvious that a lot of legacy workflows were about to look embarrassingly pre-AI. District 3's admissions research process was one of them. High-volume, repetitive, cognitively messy, and still dependent on humans to stitch together context from scattered sources. It was exactly the kind of system I wanted to rebuild, but not the kind of internal project you can casually decide to spin up on your own.

I kept thinking about that trust-calibration problem after Briefbot was done. At Saint Paul University I built an agent on the same pattern and made the question of who checks its work the centre of my research.