Python Automation Cookbook: Building AI-Driven Decision Pipelines
A practical pattern for adding lightweight AI classification steps to deterministic script pipelines, handling judgment calls that if/else logic can't cover.
This piece extends a prior Python automation cookbook by addressing how to add decision-making to script chains. The core idea is separating concerns: deterministic tasks like HTTP calls, file I/O, and scheduling stay in code, while ambiguous or unpredictable judgment calls—malformed files, unclassifiable content, unmapped edge cases—get routed through small, narrowly-scoped AI prompts instead of brittle if/elif trees.
Central to the approach is prompt_runner.py, a lightweight wrapper handling API calls, error handling, retry logic, and batch processing, designed to produce single-word classification outputs (like PROCESS/REVIEW/SKIP) that downstream code can parse reliably. The author offers a clear heuristic for choosing code versus prompts: use code when classifications are fully enumerable and inputs are structured; use prompts when input formats vary unpredictably and judgment is required.
A concrete example is a GitHub PR triage pipeline classifying 15-30 daily pull requests into AUTO, REVIEW, and URGENT categories, cutting manual triage time from 20-30 minutes to 2-3 minutes daily. AUTO classifications are accurate over 95% of the time, and URGENT classifications have had zero false negatives over six months. Cost-wise, a 100-file daily batch runs under $0.10, making the approach economically viable for production use.
For engineers, the significance lies in treating AI not as a general reasoning engine within pipelines but as a tightly-scoped classifier—a model that balances reliability and cost far better than either pure scripting or open-ended AI reasoning alone, offering a pragmatic architecture that combines both strengths.