Every AI feature ships with known flaws. The model sometimes mangles a date format, over-explains on short questions, or hedges when it should commit. Teams usually track the bugs they plan to fix. Far fewer track the ones they have *decided not to fix yet*, and that omission causes real trouble later.
A not-yet list is a short document with one entry per known failure you are consciously accepting. Each entry has four fields:
- What happens, with one concrete example input and output.
- How often, as best you can measure, even if it's a rough sample.
- Why we're accepting it: the cost of fixing outweighs the harm, or the fix risks regressions elsewhere.
- What would change our mind: a frequency threshold, a new customer segment, or a cheaper fix becoming available.
This does three useful things. First, it stops the same debate from restarting every month, because the reasoning is written down. Second, it gives support and sales an honest answer when a customer hits the problem. Third, it gives you triggers: when the metric crosses the line you wrote, the item moves from "not yet" to "now" without needing a fresh argument.
It also makes model upgrades easier. When you swap models, rerun the examples on the list. Some entries will disappear for free, and some will get worse, which is exactly the information you want before rollout.
Keep it short, review it quarterly, and delete entries that no longer reproduce. A list of accepted risks you actually revisit beats a backlog nobody reads.