Research Framework | Lab Notes — Daniel Conejo Sobrino

Phase-based research pipeline with DAG execution, artifacts, schemas, and traceable reports.

The research framework is the closest part of the ecosystem to a formal agent architecture. It models research as a pipeline with phases, dependencies, artifacts, states, and validations.

Objective

Convert a natural-language mission into a verifiable Markdown report:

Mission -> Parse -> Candidates -> Reviews / Prices / Videos -> Transcripts -> Report

Phase DAG

Loading diagram…

Research mission DAG: candidate discovery feeds reviews, pricing, video search, transcript extraction, and final reporting.

Phases

Phase	Function
F0	Create the run folder and transform the mission into `task.json`.
F1	Find candidates, normalize, deduplicate, and apply constraints.
F2	Retrieve and analyze reviews.
F4	Retrieve prices, comparisons, and seasonality.
Y1	Search for relevant videos.
Y2	Extract video transcripts.
F6	Compile the final report with sources and evidence.

The research pipeline can use an MCP-based research adapter for external information gathering. This adapter separates the pipeline from the provider-specific implementation: the pipeline requests research, receives structured results, and stores the useful evidence as artifacts.

In the current architecture, this role can be backed by a Perplexity-based MCP connector, but the important design point is the adapter boundary: external search is treated as a replaceable capability, not as logic embedded directly inside the pipeline phases.

Orchestrator

The orchestrator controls:

execution order;
parallel phases;
retries;
input validation;
partial states;
checkpoint writing;
final notification.

Phases do not call other phases. Each phase reads its inputs and writes its outputs. This separation prevents hidden flows and makes debugging easier.

Artifacts

A run creates a folder per mission:

runs/{mission-id}/
|-- task.json
|-- phase.json
|-- artifacts/
|   |-- candidates.json
|   |-- reviews.jsonl
|   |-- price_analysis.json
|   |-- youtube_urls.json
|   |-- youtube_transcripts.json
|   `-- final_report.md
|-- logs/
`-- run_summary.md

Phase States

The framework distinguishes states that are useful for real systems:

State	Meaning
`pending`	Phase has not started.
`running`	Phase is running.
`succeeded`	Phase completed successfully.
`succeeded_partial`	Output is incomplete but valid.
`failed_retryable`	Failure can be retried.
`failed_terminal`	Failure is final.
`blocked_missing_input`	Required artifacts are missing.
`skipped`	Phase skipped by design or condition.

Robustness Principle

succeeded_partial is essential. In real research, some sources fail, prices are missing, and videos do not always have transcripts. The system should not break because of that; it should document the degradation and still produce a useful report.