Kashikoi: Simulation-based evaluation for AI agents

Kashikoi is a simulation engine for benchmarking AI agents. It fits a team that wants to test an agent against customized, multi-turn scenarios and behavioral goals instead of trusting public benchmarks, hand-written prompts or a few happy-path traces.

What it does

Kashikoi’s current site ↗ lets a team connect an agent, describe what it should do and run simulated interactions that produce metrics and actionable recommendations. The examples show security, data-analysis and code agents evaluated on accuracy, context understanding, severity, false positives and task-specific success.

The useful decision is whether the benchmark reflects the product’s actual values and users. Kashikoi’s YC launch ↗ describes CPU-friendly world models that interview agents, generate diverse data and detect stale regression suites. The team says a customer can compare its agent with a competitor without writing every prompt by hand. That is an attractive promise, but the buyer still needs to inspect how scenarios are sampled, how judges are calibrated and how a score turns into an engineering change.

Kashikoi’s site displays example scores for threat detection, a data analyst and code exploration. These are product demonstrations, not independent customer results. Pricing is not public; the current path is a founder demo or waitlist-style conversation.

Founder context and tradeoffs

The YC profile identifies Aaksha Meghawat as founder and CTO and describes simulation/evaluation work at Moveworks, transformer research at CMU and edge speech models at Apple. The launch also identifies Tim Michaud as co-founder and publishes security-research context. Their background maps to the evaluation problem, but a buyer still needs to validate its own domains and failure modes.

Editorial take

I would shortlist Kashikoi for an agent team whose eval suite is stale or too expensive to maintain by hand. Start with one task and a known failure distribution. If the team cannot state what “good” means beyond a composite score, more simulation will create measurement noise rather than confidence.

Quick facts

Sources checked

Cohort context

Kashikoi is listed in Spring 2025. In our 2026-09-18 directory snapshot, 97 of 143 listed companies in that cohort have YC’s primary industry label B2B (67.8%). This is a current-directory comparison, not an original intake count or a performance ranking. Nine-cohort dataset.

Public website snapshot

Observed 2026-09-19 16:15 UTC in raw homepage HTML. This records visible metadata and advertised links, not agent execution or product quality.

Public observations · Collection method. Missing links here do not establish that a capability or file is absent elsewhere.