How Digest AI runs a round-the-clock AI news desk on free tiers
Every twenty minutes, Digest AI reads about eighty sources, turns what is new into one page per story with every source linked, checks the summaries against those sources, and publishes a static site, a daily podcast and a set of feeds. It costs nothing to run. This is how it works, and what broke on the way.
The problem
Following AI news meant opening the same story in six tabs: the lab's announcement, two press write-ups that disagree on a number, a newsletter take and a Hacker News thread. I wanted one page per event that shows what happened, who reported it, and where the reports differ, with a link to every source so nothing has to be taken on trust.
One run, start to finish
A GitHub Actions workflow starts every twenty minutes. Each run is a Python pipeline of small steps, each allowed a bounded amount of work so a slow source or a busy model never holds up the site:
- Fetch. About eighty feeds: labs and companies (their own blogs count as primary sources), the technology press, newsletters, research, and community sites. New items are stored once; a title seen before is skipped.
- Extract. The article text is pulled from each page with trafilatura. Pages that block us, or that come back with a few dozen words, are recorded as failures rather than guessed at.
- Gate. A cheap check drops what is not AI news.
- Summarise. A language model writes a headline, three key points, a short digest, why it matters, the companies, models and people named, and, for tools small businesses can use, a practical card: what it does, what it costs, how long it takes and the catch.
- Cluster. Articles about the same event are grouped into one story with small sentence embeddings (bge-small-en), so six write-ups of one launch become one page with six sources.
- Rank and export. Stories are scored and written out as JSON; an Astro build turns that into a static site on GitHub Pages, with RSS, a news sitemap and every story also as JSON and Markdown.
- Everything else. Share images, a spoken daily briefing, Bluesky posts, the admin dashboard and a morning note for me, written from the run's own numbers.
Checking a model's work
A summary is only useful if it says what the sources say. After a model writes one, a rules-only check looks for every figure and every proper name in it and confirms each appears in the source text. A summary that invents a number ("raised $3 billion" when the article says $2.5 billion) is sent back once, and if it is still wrong the offending line is removed. Headlines go through a second set of rules that strip hype ("revolutionizing", "game-changer"), clickbait frames and shouting.
When a story grows to several publishers, a stronger model rewrites it from all of them and is asked for two extra things: one sentence every report agrees on, and up to three lines on where they differ. A "differ" line is kept only if it names one of the outlets and its figures are in the articles; a model asked for disagreements will otherwise find some.
Which story leads
The daily briefing is five stories. The five are picked by score, which blends editorial importance, freshness, breadth of coverage and what readers actually open and read, learned from anonymous page events without cookies. Then one rule: a story confirmed by two publishers, or by the company itself, goes above one only a single outlet has reported. A scoop can be in the briefing; it cannot be the first thing you read while half the press is covering something else.
Running on free tiers
The whole thing costs $0 a month:
- Compute: GitHub Actions, a run of five to fifteen minutes every twenty minutes (free for a public repository).
- Hosting: GitHub Pages, a static site with no server.
- Database: Supabase's free Postgres.
- Media: share images and audio are stored as GitHub release assets.
- Voice: Kokoro, an open text-to-speech model, runs on the Actions runner.
- Language models: no single free tier covers about 550 summaries a day, so the pipeline works down a chain: Gemini first, then Ollama Cloud's free models, Groq, Cloudflare Workers AI, OpenRouter's free models and Mistral's free allowance. Each has its own daily budget, is paused for the rest of the run when it answers "too many requests", and the paid-per-token one stops itself well before its free credit runs out. A small local model on the runner is the last resort.
What went wrong
The database bill that was not a bill. Supabase's free plan allows 5 GB of data out a month. In the first cycle the pipeline used 13.4 GB, because every run re-read whole tables to find what had changed. The fix was to stop asking: the runner keeps its own copy of what it has read, Postgres triggers stamp each changed row with a revision number, and each run reads only rows newer than the last one it saw. A run now reads about 1.5 MB.
Merging too eagerly. To catch duplicates I merged stories whose summaries were very similar and named the same company. That merged different events that happened to be about the same lab on the same day. Merging is now off except for near-identical wording, and a reader was right to point it out.
A thousand files. GitHub releases take at most 1,000 assets each. Share images and thumbnails filled the week's release in five days, and every upload after that was refused. The week now carries on in a second release.
Old stories as new. Using a story's last-updated time to decide what is new put week-old stories in the briefing whenever a late article joined them. "New" now means first published.
What I would like help with
Story grouping is the weakest part: some events still split into two pages, and some near-duplicates slip through. If you notice one, or a summary that misstates its sources, the corrections address reaches me. The site is also available as data: every story is published as JSON and Markdown, documented on the API page.