hedwigaihedwigaihedwigai

Blog

Latest

SubQ against jev on biotech diligence memos: finding the sentences a column is about

To fill a typed column such as mechanism of action from a company's diligence memo, first find the sentences that state it. SubQ and jev each judged every sentence of ten memos, 8,357 questions in all, scored against sentences a person marked beforehand. Both found far more than picking whole sections: on mechanism of action, at the same length, SubQ found 57% of the marked sentences and jev 51%, against 41% for the sections. jev found more on primary endpoint, two columns tied, and the run cost about the same on each.

Read the comparison
MeasureSubQjev
Mechanism of action, found at section length57%51%
Primary endpoint, found at section length83%90%
Cost of the 8,357 questions$0.08$0.10
  • How AI harnesses work, shown on one research workbook

    A model only writes text; the harness around it runs the loop, calls the tools, manages what the model sees, saves checkpoints, sets permissions and pauses for a person. We explain each on one real workbook drafted through hedwigai's MCP server: early-stage US AI companies with a Form D in the last 30 days. The first run made 112 model calls in under 13 minutes and cited every company to its filing, and a person reading it against the filings still found 13 things to fix.

    Read how it works
    MeasureFirst runAfter review
    Model calls112120
    Wall clock12 min 39 s11 min 50 s
    Corrections a person asked for13—
  • Twenty current models, every provider, every rate

    What does a model cost? For Claude, GPT or Gemini, one price wherever you buy it, and a choice of speed: GPT-6 Astra runs from $5 to $60 per million tokens read depending on how fast you want the answer. For an open-weight model, it depends on the host: GLM-5.3 is sold by 52 companies, from $0.35 / $1.10 to Z.ai's own $1.40 / $4.40 and beyond. We priced 20 current models, newest version per family, across 84 sellers found on routers, resellers, clouds and the hosts' own price lists, and found the cheapest host flips with the job: the one that wins a job that mostly reads costs nearly four times as much on one that mostly writes.

    Read the prices
    MeasureGLM-5.3DeepSeek V4.1 FlashGPT-6 Astra
    Hosts selling it52497
    Cheapest, per 1M read / written$0.35 / $1.10$0.09 / $0.18$10 / $50
    Dearest at standard speed$2.10 / $6.60$0.65 / $1.45$12 / $60
  • Leads with a reason to write: hedwigai ask, Apollo and treg on one brief

    A cold email works when it reaches the person who decides, lands in their inbox, and opens with a reason to write today. We asked hedwigai ask, Apollo and treg for the same 40 leads at US companies that just raised money or are hiring engineers, and checked every lead the same way. ask returned 13 usable leads for $0.48, Apollo 7 for $1.90 and treg 6 for $1.40, so ask cost about a sixth as much per usable lead, and was the slowest. Why: the filing that records a raise also names the people who signed it, a database's funding filter counts loans and acquisitions, and an address labelled verified is often at a server that accepts anything.

    Read the comparison
    MeasureaskApollotreg
    Usable leads13 of 407 of 406 of 28
    Cost per usable lead$0.037$0.27$0.23
    Right person, of leads with a true reason24 of 3210 of 247 of 21
  • Time machine: leak-free RL environments, graded by what actually happened

    Time machine is a feature for hedwigai: pick a date, and every source your research reads answers as it would have on that day, with later filings hidden and prices as printed then. It serves two uses: primary research as it could have been done at the time, and reinforcement-learning environments for forecasting, where an agent bets on what came next and is scored against what has since happened. We simulated it first on EDGAR: 15,277 bets on whether a company's next public float rises, over four years. An RL player trained on payouts alone couldn't beat a coin, and a clock set one season late beat everything, without a single unpublished filing.

    Read the design
    MeasureRL playerA coin
    Score, clock before the answer is measured0.7470.750
    Score, clock after it (a leak)0.7620.750
    Bets simulated, 2022 to 202515,27715,277
  • Self-hosted decisions for ask: Perplexity's open Decider in place of jev on 92 company questions

    Inside hedwigai ask, a decision model makes the small calls on the way to an answer: which lookups to run, which figure in a filing is the amount in question, and the final yes or no. We swapped jev, TypeSafe's decision model, for Perplexity's open Decider running on two NVIDIA RTX 3090 GPUs (24 GB of VRAM each), and put the same 92 questions about public companies to both, checked against SEC filings. Both got 89 right; Decider got one more wrong and was about nine times slower (19.1 s a question against 2.1 s). One sentence in a lookup's instructions changed 22 of Decider's answers.

    Read the comparison
    Measureask with jevask with Decider
    Questions right, of 928989
    Wrong answers12
    Median time per question2.1 s19.1 s
    Where the decisions runTypeSafe's jev servicetwo NVIDIA RTX 3090s, 24 GB each
  • Company lists in ask: a small model trained to put the right companies first, tested on 33 new questions

    Ask hedwigai ask for a list of companies, like food distributors in Goa, and it answers with ten, each with the sentence and page it came from. A small model decides which ten. We trained one for the job: on 33 questions it had never seen, 62% of the companies in its top tens were correct, up from 55%, with better lists for 18 questions and worse for 5. Along the way we found a mistake our two AI markers shared, counting the brands a distributor sells as distributors, and how we fixed it.

    Read how it works
    MeasureTrained for listsGeneral-purpose model
    Correct, of the companies in the top 10s62%55%
    Correct companies in the top 10s142117
    Wrong companies in the top 10s8697
  • Clinical decisions: jev and ask on System One, PubMedQA, SciFact and a 370-decision set

    jev, TypeSafe's decision model, which can be used as the model in hedwigai ask, on clinical decisions. On EdgeEvals' System One, a public leaderboard that gives 29 models the same 669 private clinical decisions, jev ranks second: 93.9% right, one severe error. On two open medical tests, PubMedQA and SciFact, jev and ask both beat the best model EdgeEvals scored (on PubMedQA 77.0% and 74.4% against 62.2%). On 370 decisions we wrote to System One's rules, each answer confirmed by a second reader, jev was right on 94.6% with one severe error and ask on 93.0% with none.

    Read the comparison
    Measurejevask
    Our 370 clinical decisions, right94.6%93.0%
    PubMedQA, right (best EdgeEvals scored: 62.2%)77.0%74.4%
    System One board, of 29 models2ndnot entered
  • Executive lookup for US-listed companies: Claude Opus 5.5 and ask on 95 questions

    Who is the chief executive, finance chief or general counsel of a US-listed company, and what is their email? We put 95 such questions, some about companies that don't exist, to Claude Opus 5.5 answering from memory and to hedwigai ask, which looks each one up, and checked them against SEC filings. Claude got 63 right and 4 wrong, strongest on large companies. ask got 59 right and 11 wrong, did better on small companies, and was the only one to find email addresses (5 of 15).

    Read the comparison
    MeasureaskClaude Opus 5.5
    Questions right, of 955963
    Small companies right, of 20139
    Emails found, of 1550
  • Rental comps: floor-plan rents and amenities on 14 listings, five assistants compared

    To price a rental you compare it with similar listings: the rent for each floor plan, and the amenities. We gave the same 14 listings to hedwigai's realestate-us and to Claude, GPT, Gemini and Perplexity with web search, and checked every answer against the listing page. realestate-us got all 26 floor-plan rents right (the best assistant 23) and claimed 12 amenities the listing doesn't mention (the assistants 129 to 185), in 1.8 s a listing for under 4 cents.

    Read the comparison
    Measurerealestate-usBest of four assistants
    Floor-plan rents right, of 262623
    Amenities claimed the listing doesn't state12129
    Time per listing1.8 s7.5 s
  • Company screening from SEC filings: jev, Vela 2.0 and ask on 92 yes/no questions

    92 yes-or-no questions about public companies, such as whether a company is a biotech whose shares held by outside investors are worth more than a given amount, checked against SEC records. jev answering on its own got 73 right and 19 wrong; Vela 2.0, an open decision model, got 43 right. hedwigai ask, which reads the filings, does the arithmetic in code and cites its sources, got 88 right, 2 wrong, and said it was unsure on 2. The post also shows how to add ask to your assistant.

    Read the comparison
    Measureaskjev
    Right, of the questions answered98%79%
    Wrong answers, of 92219
    Median time, a new question1.07 s0.53 s
  • BloomFrameAI for Petal+Eon

    Petal+Eon hosts floral workshops where people press real preserved flowers into a BloomFrame. BloomFrameAI turns a text prompt or an uploaded photo into custom artwork for it — within the frame's margins, around the flower area, and ready for the die cutter — so every frame can be personal without being made by hand.

    Read the story
    Workshops hosted1,000+
    Frame constraints met5
    Stages per design3
    CustomerPetal+Eon
  • A same-model decider above an agent's writer: one task, two configurations

    An AI agent writing a document usually has one model choose each next step. We tried putting a second call to the same model above it, a "decider" that picks each step for the writer, and ran one research task both ways. With the decider, the run made 3.1 times as many model calls, read 10.7 times as much text and took six times as long, and the document came out the same, down to the same three email addresses that bounce. One task, one run each, and what we changed as a result.

    Read the comparison
    MeasureWith a deciderWithout
    Wall clock23m 36s3m 46s
    Model calls10434
    Prompt tokens6.62M620k
    Document producedthe samethe same
  • Biotech due-diligence document gathering: coverage across 11 companies

    Before anyone invests in or partners with a biotech, someone gathers its documents: SEC filings, FDA drug labels, clinical-trial registrations, investor presentations and published papers. We gave that job to a hedwigai research run for 11 companies and checked each document it wrote against a checklist written before the run. On average it covered 97% of the checklist.

    Read the numbers
    Companies covered11
    Mean coverage, 0 to 10.971
    Wall clock3h 11m
    Tokens55.9M in · 241k out
  • US importer search for heavy engineering equipment: coverage across 5 product lines

    Finding US companies that import heavy engineering equipment, and checking whether each is a real lead: US customs shipping records, utility purchasing, awards for large construction projects, and the certifications and import duties that decide whether a sale can happen. A hedwigai research run did this for five product lines; against a checklist written before the run, it covered 96% on average.

    Read the numbers
    Product lines covered5
    Mean coverage, 0 to 10.956
    Wall clock35m 35s
    Tokens6.67M in · 76k out
  • Aircraft records completeness: an evaluation method (no models scored yet)

    When an aircraft is sold or comes back from a lease, its paperwork has to prove its history: the life used on each part that has a hard limit, every mandatory safety fix, approved repairs, release certificates, and the lease's own return terms. This post sets out how we will test whether an AI can tell that paperwork is complete, with a worked example of a scored result. No model has been tested on it yet.

    Read the method
    Methods described8
    Kinds of defect9
    Defects seeded per package82
    Test materialsynthetic · NDA gold set
  • Defect localization in industrial images: four frontier models and a trained localizer

    Factories inspect welds, castings and parts from photos. The largest general AI models can describe a defect, but can they mark where it is? Four of them drew one box per image. We compared them with a rule that never looks at the image and with a small model trained only for this job. The trained model's boxes overlapped the real defect about four times as much (0.421 against 0.109, on a scale from 0 to 1).

    Read the numbers
    MeasureTrained localiserBest of four frontier models
    Mean overlap with the defect (IoU, 0 to 1)0.4210.109