Amon Taboi is the founder of Crescevo and editor of Crescevo AI. He builds Crescevo's AI content and automation pipeline and writes on shipping LLM systems in production — RAG, evaluations, and agent tooling.
ai
TL;DR * AI products can fail in systematically biased ways rather than randomly distributed, with 0% guarantee of throwing an error when a bug occurs, unlike traditional software products. * Every AI feature that produces output a user acts on needs an explicit confidence threshold to determine when to abstain or
ai
TL;DR * An agent making 15 LLM calls to complete a task requires observability infrastructure to identify which call introduced an error and whether that error is reproducible. * Generating a UUID at the entry point of each agent execution and passing it through every sub-call is the most important investment
ai
TL;DR * Fine-tuning a large model costs thousands of dollars and weeks of work, making it a costly mistake if not used appropriately for a specific use case. * A model fine-tuned on a specific dataset, such as product documentation, can start giving stale answers in as little as 2 months,
ai
TL;DR * Collecting 200 to 500 production-representative inputs is recommended for task-specific evaluation design to ensure the model performs well on actual task requirements. * Large models have likely seen test set contents in their training data, which can lead to benchmark scores measuring memorization rather than generalization. * Models that score
ai
TL;DR * Mistral 7B, Llama 3.1 8B, or Qwen2.5 7B models perform well for structured generation tasks in a local LLM setup. * The core workflow architecture in this content automation pipeline consists of five nodes: Trigger, signal collection, LLM generation, post-processing, and publishing or review queue. * Ollama runs
ai
TL;DR * Three failure modes, specifically chunking strategy mismatch, embedding model mismatch, and retrieval scoring calibration, account for the majority of production Retrieval-Augmented Generation failures. * Most RAG implementations use fixed-length chunking, typically splitting the document every 512 tokens, which works adequately for short-form content but fails for long-form technical documentation.
ai
TL;DR * Most teams adopting LLMs in production hit a ceiling at around 60 to 70 percent of the model's actual capability due to prompting limitations rather than model limitations. * Fixing the three structural errors in prompting can recover 30 to 40 percent of the performance gap without
jobs
Related reading * Uber's $1,500/month AI limit is a useful signal for AI tool pricing * DuckDuckGo search saw 28% more visits after Google said people love AI mode * I think Anthropic and OpenAI have found product-market fit Before you go — a free checklist Shipping an AI feature?
newsletter
TL;DR * Claude's extended thinking mode exposes a genuine scratchpad, allowing users to see the model's internal deliberation and reasoning path before arriving at a conclusion, which is particularly useful for tasks like legal or financial analysis. * The quality ceiling for hard problems is materially higher
newsletter