Evidence · Two-week Pilot

Same 10 questions. Two paths.
Every intervention stays in the record.

A polished answer is not proof. Freeze the questions, current path, reviewer, and rubric first. Then compare Spotonix without moving the goalposts or hiding the failures.

Book DemoSee how it works

Same data slice · Same frozen questions · Same reviewer · Same rubric

Before either path runs

Freeze the test.
Then earn the result.

A Two-week Pilot starts when the environment is data-ready and the comparison rules are fixed—not when the first meeting ends.

01

10 real questions

Freeze them before the Spotonix run: ordinary questions, expert questions, unseen follow-ups, one ambiguity case, and one repeat or paraphrase.

02

One bounded domain

Use a focused data slice that can be made ready without pretending the whole company is already connected.

03

Your current path

Include the tools, handoffs, review, repair, and exception handling your team needs to make an answer defensible today.

04

One reviewer and rubric

Agree what each question should do—and which measure matters—before either path is scored.

Data-ready means: a permitted access path, one bounded domain, working credentials, a supported model or YAML/SQL reference, and frozen questions, reviewer, and rubric. Results within two weeks of data-ready.

The comparison

The baseline includes
the human trust work.

A real current path is not just a model or dashboard. It includes the person who checks the meaning, validates the source, repairs the logic, and decides whether anyone should use the answer.

Path A · Baseline

Your current path

The tools and people your team relies on today—including the work required to decide whether the answer is safe to use.

  1. 01

    QuestionThe frozen business question

  2. 02

    Drafts + handoffsTools, queries, reports, and people

  3. 03

    Human verification + repairCheck meaning, source, logic, and exceptions

  4. 04

    Reviewer decisionUse, clarify, refuse, repair, or fail

Human trust work is part of the baseline, not invisible overhead.

Path B · Spotonix

Inspectable product path

Run the identical set without changing the reviewer, expected disposition, or what counts as right.

  1. 01

    QuestionThe identical frozen question

  2. 02

    Interpret + show the planAsk where company meaning is missing

  3. 03

    Logic + sourcesExpose the basis and supporting artifacts

  4. 04

    Reviewer decisionUse, clarify, refuse, repair, or fail

The product exposes the path; the agreed rubric determines whether it helped.

Same questions · Same reviewer · Same rubric

InterpretationRetriesExpert workTime to defensible answerReviewer dispositionEvery failure

The frozen set

Ordinary questions.
Materially different readings.

The highlighted concept is where an ordinary question can split into different populations, measures, or windows. These examples span SaaS, AdTech, FinTech, and GTM Ops; your actual 10 are frozen before the run.

Needs your definition Business concept under interpretation

The highlight marks meaning the reviewer may need to define—not an error and not a claim that Spotonix already knows the answer.

  1. 01

    SaaS

    How many active customers did we have last month?

    “Active” can mean a login, a key action, or account-level activity. Each reading changes the population and unit.

    See the clarification
    Illustrative product moment Ask · Choose · Show
    You asked

    How many active customers did we have last month?

    Needs your definition

    Clarification requested by Spotonix

    Which definition should this analysis use?

    Plan before execution

    Activity
    At least one login Defined by your team Changes with your choice
    Unit
    Distinct users Assembled by Spotonix Changes with your choice
    Window
    Previous calendar month Defined inline

    The selected definition can change both the activity rule and what gets counted. Answering without asking means making that choice invisibly.

  2. 02

    SaaS

    What is recognized revenue this quarter?

    Recognized, billed, booked, and collected amounts belong to different calendars and source systems.

    See the clarification
    Illustrative product moment Ask · Choose · Show
    You asked

    What is recognized revenue this quarter?

    Needs your definition

    Clarification requested by Spotonix

    Which definition should this analysis use?

    Plan before execution

    Basis
    Recognized revenue, net of credits Defined by your team Changes with your choice
    Total
    Summed amount Assembled by Spotonix
    Calendar
    Fiscal quarter Found in your model Changes with your choice

    The selected definition changes the plan before any query runs. Answering without asking means making that choice invisibly.

  3. 03

    GTM Ops

    Show me our top accounts.

    The ranking changes when “top” means recent revenue, product usage, or pipeline—and when child accounts roll up.

    See the clarification
    Illustrative product moment Ask · Choose · Show
    You asked

    Show me our top accounts.

    Needs your definition

    Clarification requested by Spotonix

    Which definition should this analysis use?

    Plan before execution

    Rank by
    Recognized revenue Defined by your team Changes with your choice
    Window
    Trailing 90 days Defined inline Changes with your choice
    Unit
    Parent account Found in your model Changes with your choice

    The selected definition changes the plan before any query runs. Answering without asking means making that choice invisibly.

  4. 04

    SaaS

    What is our churn rate?

    Logo churn and gross revenue churn use different populations, measures, and denominators.

    See the clarification
    Illustrative product moment Ask · Choose · Show
    You asked

    What is our churn rate?

    Needs your definition

    Clarification requested by Spotonix

    Which definition should this analysis use?

    Plan before execution

    Population
    Customers at period start Found in your model Changes with your choice
    Measure
    Lost customer count Assembled by Spotonix Changes with your choice
    Denominator
    Opening customers Found in your model Changes with your choice

    The selected definition changes the plan before any query runs. Answering without asking means making that choice invisibly.

  5. 05

    AdTech

    Which campaigns are converting best?

    The conversion event, attribution model, window, and choice of rate versus volume can all change the order.

    See the clarification
    Illustrative product moment Ask · Choose · Show
    You asked

    Which campaigns are converting best?

    Needs your definition

    Clarification requested by Spotonix

    Which definition should this analysis use?

    Plan before execution

    Event
    Configured conversion event Found in your model
    Credit
    Last-touch attribution Defined by your team Changes with your choice
    Rank by
    Conversion rate Assembled by Spotonix Changes with your choice

    The selected definition changes the plan before any query runs. Answering without asking means making that choice invisibly.

  6. 06

    AdTech

    How many billable impressions did we deliver?

    Served, measured, viewable, invalid-traffic-filtered, and contract-billable impressions are not interchangeable.

    See the clarification
    Illustrative product moment Ask · Choose · Show
    You asked

    How many billable impressions did we deliver?

    Needs your definition

    Clarification requested by Spotonix

    Which definition should this analysis use?

    Plan before execution

    Event
    Ad server impression Defined by your team Changes with your choice
    Filter
    Known invalid traffic excluded Defined by your team Changes with your choice
    Source
    Ad server log Found in your model Changes with your choice

    The selected definition changes the plan before any query runs. Answering without asking means making that choice invisibly.

  7. 07

    FinTech

    What is our collection efficiency this month?

    Collected cash divided by current-period dues is different from cash collected against opening receivables.

    See the clarification
    Illustrative product moment Ask · Choose · Show
    You asked

    What is our collection efficiency this month?

    Needs your definition

    Clarification requested by Spotonix

    Which definition should this analysis use?

    Plan before execution

    Measure
    Cash collected on current dues Assembled by Spotonix Changes with your choice
    Denominator
    Amount due this month Found in your model Changes with your choice
    Scope
    Calendar month Defined by your team Changes with your choice

    The selected definition changes the plan before any query runs. Answering without asking means making that choice invisibly.

  8. 08

    FinTech

    How many bad loans do we have?

    Delinquent accounts, non-performing loans, and charged-off principal are different business objects.

    See the clarification
    Illustrative product moment Ask · Choose · Show
    You asked

    How many bad loans do we have?

    Needs your definition

    Clarification requested by Spotonix

    Which definition should this analysis use?

    Plan before execution

    Rule
    90+ days past due Defined by your team Changes with your choice
    Measure
    Count of loan accounts Assembled by Spotonix Changes with your choice
    As of
    Period end Defined inline Changes with your choice

    The selected definition changes the plan before any query runs. Answering without asking means making that choice invisibly.

  9. 09

    GTM Ops

    How much qualified pipeline do we have?

    Qualification may come from an accepted sales handoff or a CRM stage, and pipeline may be raw or probability weighted.

    See the clarification
    Illustrative product moment Ask · Choose · Show
    You asked

    How much qualified pipeline do we have?

    Needs your definition

    Clarification requested by Spotonix

    Which definition should this analysis use?

    Plan before execution

    Entry
    Sales-accepted opportunity Found in your model Changes with your choice
    Amount
    Unweighted amount Assembled by Spotonix Changes with your choice
    Scope
    Open opportunities Found in your model

    The selected definition changes the plan before any query runs. Answering without asking means making that choice invisibly.

  10. 10

    FinTech

    How many transactions were flagged as suspicious last week?

    A threshold alert, a model alert, and an investigator disposition represent different points in the workflow.

    See the clarification
    Illustrative product moment Ask · Choose · Show
    You asked

    How many transactions were flagged as suspicious last week?

    Needs your definition

    Clarification requested by Spotonix

    Which definition should this analysis use?

    Plan before execution

    Event
    Threshold or model alert Defined by your team Changes with your choice
    Count
    Distinct transactions Assembled by Spotonix
    Window
    Alert created last week Defined inline Changes with your choice

    The selected definition changes the plan before any query runs. Answering without asking means making that choice invisibly.

Why asking is the product

Unresolved meaning is an explicit
product state.

A general model may choose a plausible meaning—or ask if prompted. Spotonix makes unresolved meaning an explicit product state: it pauses, asks, records the team’s choice, and shows the resulting plan before execution.

Internal engineering note · July 2026

Latency, not proof
of correctness.

These measurements help set expectations for an evaluated run. They are not customer outcomes, a competitive benchmark, or evidence that an answer was right.

30.7s

Interpretation + plan ready

Median; 18.1–41.0 seconds across 12 first-time questions.

54.2s

Query ready

Median; 27.0–61.7 seconds. 11 of 12 reached this milestone within 60 seconds.

70.3s

Full response persisted

Median; 37.4–126.1 seconds across the same set.

Method

12 first-time questions, locally executed

Two anonymized July 2026 internal runs against a local DuckDB environment, concurrency 1. Customer and workspace identifiers have been removed. Three exact-match cache replays were excluded from the 12-question set.

Boundary

Correctness is unmeasured

The retained ground-truth set contains no source cases, so these runs cannot support an accuracy or correctness claim. The Two-week Pilot uses your named reviewer and frozen rubric to create that record.

One failure, kept in the record

Transparency is a record.
Not a promise.

This historical internal evaluation did not produce a final answer. The retained artifact makes the failure visible without converting it into a success.

Historical internal evaluation · May 19, 2026

The run stopped before an answer—and stayed a failure in the record.

The question, as evaluated

Among the bottom 5 performing stores by profit last quarter, show the month-over-month trend for each of the last 3 months, their prior quarter numbers, and flag stores where profit dropped more than 20% using a threshold classify.

What happened
The SQL agent timed out on its first tool call. No tool call completed and no answer was produced.
What was recorded
90.3 seconds elapsed · tool_calls=0
How it was scored
A failure. Nothing was retried, repaired, or graded as a success.

Not in the artifact: a reviewer disposition, a product-change record, or a retest outcome. Those fields stay empty rather than invented.

What the buyer receives

An Answer Evidence Pack.
Not a highlight reel.

Every question carries enough context to understand what happened, how much work it required, and why the reviewer accepted, challenged, or rejected it.

Question scorecard

Expected and actual disposition, reviewer decision, severity, and rationale for every question.

Answer basis

Interpretation, plan provenance, generated logic or SQL when present, sources, and supporting result artifacts.

Effort log

Retries, reformulations, expert interventions, role, minutes, review, and repair for both paths.

Gap map + decision

Every miss classified, the comparative delta shown with limitations, and a go, fix-and-retest, or stop recommendation.

Before Day 1Data ready

Permitted access, credentials, one bounded domain, a supported model or YAML/SQL reference, questions, reviewer, and rubric are confirmed.

Within two weeks of data-readyRun both paths

Record answers, clarifications, retries, expert work, and failures.

ReadoutMake the decision

Compare the evidence and choose go, fix-and-retest, or stop.

Go, fix-and-retest, or stop—the buyer keeps the Answer Evidence Pack either way.

A Pilot can fail. That is part of the proof.

If your current path is better on the questions that matter, Spotonix should not win.

Bring the questions, your current path, and the person who decides what counts as right.

Book DemoTalk to us first