{"id":950,"date":"2026-10-03T09:12:17","date_gmt":"2026-10-03T09:12:17","guid":{"rendered":"https:\/\/perit.ai\/blogs\/?p=950"},"modified":"2026-10-03T09:12:17","modified_gmt":"2026-10-03T09:12:17","slug":"when-ai-gets-it-right-most-of-the-time-the-problem-with-rare-cases","status":"publish","type":"post","link":"https:\/\/perit.ai\/blogs\/when-ai-gets-it-right-most-of-the-time-the-problem-with-rare-cases\/","title":{"rendered":"When AI Gets It Right Most of the Time: The Problem With Rare Cases"},"content":{"rendered":"

The Data Distribution Is Not the Real World<\/h2>\n

A dataset does not become representative just because it is large. A model can be trained on millions of examples and still have very little experience with the situations it will eventually face in the real world.<\/p>\n

Consider an autonomous driving system trained on millions of hours of road footage. Most of that data might show ordinary daytime driving: clear roads, visible lane markings, familiar intersections, and predictable traffic. But only a small portion may contain situations such as a pedestrian suddenly stepping out from behind a parked vehicle, heavy rain obscuring lane markings, or a cyclist appearing from an unexpected direction. These situations may be rare in the dataset, but they can matter enormously when the system encounters them on the road.<\/p>\n

This creates a gap between what the model sees most often and what it might actually encounter<\/strong>. AI systems learn from the patterns present in their training data, so situations that appear rarely\u2014or not at all\u2014give the model much less experience to learn from. NIST research on data coverage has also examined how the frequency of different input combinations can affect AI performance. [1]<\/a><\/p>\n

The gap can become even wider after deployment. The people, environments, behaviors, and conditions a model encounters in the real world are not always the same as those represented in its training data. A model might perform well on a carefully prepared test set and still struggle when it encounters situations that were barely represented during training. [2]<\/a><\/p>\n

\n

Most Training Data Lives in Common Cases<\/h3>\n
\n
<\/div>\n
<\/div>\n
<\/div>\n
<\/div>\n
<\/div>\n
<\/div>\n
<\/div>\n
<\/div>\n
<\/div>\n
<\/div>\n
<\/div>\n
<\/div>\n<\/div>\n
Common cases<\/strong>
\nRare \/ long-tail cases<\/strong><\/div>\n

Large datasets often contain many examples of familiar situations,
\nwhile unusual scenarios make up a much smaller part of the distribution.<\/p>\n<\/div>\n

A large dataset gives an AI system more examples to learn from. It does not necessarily give the model the right examples.<\/strong><\/p>\n

Why Rare Cases Are So Difficult to Learn<\/h2>\n

Rare cases are difficult for AI systems for a simple reason: the model just does not get to see them often enough.<\/p>\n

If a model sees thousands of examples of the same situation, it has plenty of opportunities to learn what matters. But unusual situations might appear only a few times in the entire training set. And when they do appear, they may not look exactly alike. One example might involve poor lighting, another an unusual camera angle, and another a combination of several conditions the model has never encountered together.<\/p>\n

Take an autonomous driving system. It can see millions of examples of normal roads, pedestrians, traffic signals, and vehicles. But what happens when heavy rain, poor visibility, an unexpected obstacle, and an unusual road layout occur at the same time? Each of these conditions may be familiar on its own, but their combination may be almost absent from the training data.<\/p>\n

This is what makes the long tail<\/strong> difficult. [3]<\/a> A relatively small number of common situations account for most of the data, while a much larger number of unusual situations appear only occasionally. The model gets plenty of practice with the first group and very little with the second.<\/p>\n

\n
A simple way to think about representation<\/div>\n
Rare-case representation = Rare examples \u00f7 Total examples
\n<\/span><\/div>\n

When rare situations make up only a small fraction of the training data,
\nthe model gets far fewer opportunities to learn how those situations behave.<\/p>\n<\/div>\n

There is another complication: rare cases are often not simply new categories. They can be new combinations<\/strong> of things the model already knows. A system might correctly recognize a pedestrian, a vehicle, and a road obstruction individually, yet still struggle when all three appear together in an unfamiliar situation.<\/p>\n

That is why rare does not necessarily mean unimportant. A case can appear only a handful of times in a dataset and still become one of the most important cases to handle correctly once the AI is deployed in the real world.<\/p>\n

When Accuracy Hides the Problem<\/h2>\n

One of the easiest ways to miss rare-case failures is to look at the overall accuracy of a model.<\/p>\n

Imagine a system tested on 100,000 examples. If 98,000 of those examples represent common situations and the model performs extremely well on them, it can still achieve an impressive overall score even if it performs poorly on the remaining 2,000 rare cases. The final percentage looks good, but it does not tell us where the model is struggling.<\/p>\n

\n\n\n\n\n\n\n\n
Scenario<\/th>\nExamples<\/th>\nModel Performance<\/th>\n<\/tr>\n<\/thead>\n
Common situations<\/td>\n98,000<\/td>\n99%<\/td>\n<\/tr>\n
Rare situations<\/td>\n2,000<\/td>\n70%<\/td>\n<\/tr>\n
Overall<\/td>\n100,000<\/td>\n98.42%<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

A strong overall score can still hide substantially weaker performance on the cases that appear least often.<\/p>\n

This matters because the cases that are hardest to find in a dataset can sometimes be the ones that matter most in practice. An autonomous driving system, for example, may handle ordinary roads almost perfectly while still struggling with an unusual road layout or an unexpected pedestrian movement. Those situations might have little effect on its average accuracy simply because they occur so infrequently.<\/span><\/p>\n

This is why a single benchmark score rarely tells the whole story.\u00a0 [4]<\/a> Teams need to look at performance across different scenarios, conditions, and types of data, especially where the model has fewer examples to learn from.<\/span><\/p>\n

The question is therefore not just <\/span>\u201cHow accurate is the model?\u201d<\/strong> It is also <\/span>\u201cWhere does that accuracy break down?\u201d<\/strong><\/p>\n

Finding those weak spots is often the first step toward discovering what the training data is missing.<\/span><\/p>\n<\/div>\n

Finding the Cases the Model Has Missed<\/h2>\n

Once a model’s weak spots become visible, the next challenge is figuring out what is missing from the data. Simply adding more examples is not always the answer. If the new data looks almost identical to what the model already knows, the dataset becomes larger without becoming much more useful.<\/p>\n

A better approach is to look at where the model repeatedly struggles. For an autonomous driving system, that could mean collecting more examples from heavy rain, poorly lit roads, unusual intersections, or situations where pedestrians and vehicles behave unpredictably. For a voice system, it could mean looking at uncommon accents, background noise, overlapping speech, or phrases that the model regularly misinterprets.<\/p>\n

This turns data collection into a more targeted process.\u00a0 [5]<\/a> Instead of asking \u201cHow can we get more data?\u201d<\/strong>, teams can ask \u201cWhat kind of data would help the model handle this failure better?\u201d<\/strong><\/p>\n

\n

From Model Failure to Targeted Data<\/h3>\n
\n

<\/p>\n

Model Failure<\/div>\n
\u2193<\/div>\n

<\/p>\n

Identify Pattern<\/div>\n
\u2193<\/div>\n

<\/p>\n

Find Data Gap<\/div>\n
\u2193<\/div>\n

<\/p>\n

Collect Targeted Data<\/div>\n
\u2193<\/div>\n

<\/p>\n

Review & Annotate<\/div>\n<\/div>\n<\/div>\n

Production data can be particularly valuable here. Every unexpected prediction, correction, or failure can reveal a situation that was poorly represented during training. Those examples can then be reviewed, labelled, and added back into the training or evaluation process where appropriate.<\/p>\n

The goal is not to collect every possible situation. That would be impossible. The goal is to continuously identify the gaps that matter and make sure the dataset becomes better at representing them over time.<\/p>\n

More Data Is Not Always Better Data<\/h2>\n

Once a gap has been identified, the obvious response is often to collect more data. But adding data indiscriminately can make a dataset larger without making it more useful.<\/p>\n

What matters is what the new data adds<\/strong>. If an autonomous driving system already has millions of examples of clear-weather highway driving, collecting another million similar examples will do little to prepare it for heavy rain, poorly marked roads, or pedestrians appearing from unexpected locations. A smaller, carefully selected set of those missing scenarios could provide much more useful coverage. [6]<\/a><\/p>\n

This is why data collection increasingly needs to be guided by the behaviour of the model itself. Teams can use failed predictions, production feedback, and evaluation results to identify which situations deserve more attention. The aim is not simply to increase the number of examples, but to improve the range of situations the model has learned to handle.<\/p>\n

The same principle applies to annotation. Rare and difficult examples often require more careful review because a small labelling mistake can make an already limited signal even harder for the model to learn from. Human experts can help determine which cases are genuinely important, how they should be labelled, and whether similar examples should be added to the dataset.<\/p>\n

\n

The Continuous Data Improvement Loop<\/h3>\n
\n

<\/p>\n

Deploy<\/div>\n
\u2193<\/div>\n

<\/p>\n

Observe<\/div>\n
\u2193<\/div>\n

<\/p>\n

Evaluate<\/div>\n
\u2193<\/div>\n

<\/p>\n

Improve Data<\/div>\n
\u2193<\/div>\n

<\/p>\n

Retrain<\/div>\n
\u2193<\/div>\n

<\/p>\n

Test Again<\/div>\n
Feed the results into the next improvement cycle<\/div>\n<\/div>\n<\/div>\n

In other words, the value of data depends not only on how much you have, but on what it teaches the model that it did not already know.<\/strong><\/p>\n

What Reliable AI Looks Like Beyond the Average<\/h2>\n

AI systems will always have cases they have never seen before. The goal is not to build a dataset that contains every possible situation, because no dataset can do that. The goal is to understand where the gaps are and keep reducing the ones that matter.<\/p>\n

That means looking beyond overall accuracy and paying attention to the situations where models struggle. A rare example may seem insignificant when viewed on its own, but a collection of these examples can reveal a pattern that a conventional benchmark would miss.<\/p>\n

For teams building AI for real-world environments, this makes data quality a continuous process. Models need to be evaluated against changing conditions, their failures need to be investigated, and the resulting insights need to feed back into data collection and training.<\/p>\n

\n

The long tail is therefore not simply a problem of having too little data.<\/strong>
\nIt is a reminder that reliable AI depends on having the right data for the
\nsituations that matter\u2014even when those situations are rare.<\/p>\n<\/div>\n

References<\/h2>\n
\n

[1] <\/strong>Data Frequency Coverage Impact on AI Performance
\n<\/a>Erin Lanus, Brian Lee, Jaganmohan Chandrasekaran, Laura Freeman,
\nM S Raunak, Raghu Kacker, David Kuhn. NIST, 2025.<\/p>\n

[2] <\/strong>WILDS: A Benchmark of in-the-Wild Distribution Shifts
\n<\/a>Pang Wei Koh et al. Proceedings of the 38th International Conference
\non Machine Learning, PMLR 139, 5637\u20135664, 2021.<\/p>\n

[3] <\/strong>Long-tailed Visual Recognition with Deep Models: A Methodological Survey and Evaluation
\n<\/a>Yu Fu, Liuyu Xiang, Yumna Zahid, Guiguang Ding, Tao Mei,
\nQiang Shen, Jungong Han. Neurocomputing, 509, 290\u2013309, 2022.<\/p>\n

[4] <\/strong>Data Imbalance in Classification: Experimental Evaluation
\n<\/a>Science of Computer Programming, 2020.<\/p>\n

[5] <\/strong>Active Learning for Data Streams: A Survey
\n<\/a>Davide Cacciarelli, Murat Kulahci. Machine Learning, 113,
\n185\u2013239, 2024.<\/p>\n

[6] <\/strong>Data-Centric Artificial Intelligence
\n<\/a>Johannes Jakubik, Michael V\u00f6ssing, Niklas K\u00fchl, Jannis Walk,
\nGerhard Satzger et al. Business & Information Systems Engineering,
\n66, 507\u2013515, 2024.<\/p>\n<\/div>\n","protected":false},"excerpt":{"rendered":"

The Data Distribution Is Not the Real World A dataset does not become representative just because it is large. A model can be trained on millions of examples and still\u2026<\/p>\n","protected":false},"author":1,"featured_media":953,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[22],"tags":[],"class_list":["post-950","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-artificial-intelligence"],"yoast_head":"\nWhen AI Gets It Right Most of the Time: The Problem With Rare Cases - Perit Blog<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/perit.ai\/blogs\/when-ai-gets-it-right-most-of-the-time-the-problem-with-rare-cases\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"When AI Gets It Right Most of the Time: The Problem With Rare Cases - Perit Blog\" \/>\n<meta property=\"og:description\" content=\"The Data Distribution Is Not the Real World A dataset does not become representative just because it is large. A model can be trained on millions of examples and still\u2026\" \/>\n<meta property=\"og:url\" content=\"https:\/\/perit.ai\/blogs\/when-ai-gets-it-right-most-of-the-time-the-problem-with-rare-cases\/\" \/>\n<meta property=\"og:site_name\" content=\"Perit Blog\" \/>\n<meta property=\"article:published_time\" content=\"2026-10-03T09:12:17+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/10\/When-AI-Gets-It-Right.webp\" \/>\n\t<meta property=\"og:image:width\" content=\"1672\" \/>\n\t<meta property=\"og:image:height\" content=\"941\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/webp\" \/>\n<meta name=\"author\" content=\"Kshitij@Perit\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Kshitij@Perit\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"8 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/when-ai-gets-it-right-most-of-the-time-the-problem-with-rare-cases\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/when-ai-gets-it-right-most-of-the-time-the-problem-with-rare-cases\\\/\"},\"author\":{\"name\":\"Kshitij@Perit\",\"@id\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/#\\\/schema\\\/person\\\/954939b1549e8f3089b46850881d9775\"},\"headline\":\"When AI Gets It Right Most of the Time: The Problem With Rare Cases\",\"datePublished\":\"2026-10-03T09:12:17+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/when-ai-gets-it-right-most-of-the-time-the-problem-with-rare-cases\\\/\"},\"wordCount\":1702,\"commentCount\":0,\"image\":{\"@id\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/when-ai-gets-it-right-most-of-the-time-the-problem-with-rare-cases\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/wp-content\\\/uploads\\\/2026\\\/10\\\/When-AI-Gets-It-Right.webp\",\"articleSection\":[\"Artificial Intelligence\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/perit.ai\\\/blogs\\\/when-ai-gets-it-right-most-of-the-time-the-problem-with-rare-cases\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/when-ai-gets-it-right-most-of-the-time-the-problem-with-rare-cases\\\/\",\"url\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/when-ai-gets-it-right-most-of-the-time-the-problem-with-rare-cases\\\/\",\"name\":\"When AI Gets It Right Most of the Time: The Problem With Rare Cases - Perit Blog\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/when-ai-gets-it-right-most-of-the-time-the-problem-with-rare-cases\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/when-ai-gets-it-right-most-of-the-time-the-problem-with-rare-cases\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/wp-content\\\/uploads\\\/2026\\\/10\\\/When-AI-Gets-It-Right.webp\",\"datePublished\":\"2026-10-03T09:12:17+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/#\\\/schema\\\/person\\\/954939b1549e8f3089b46850881d9775\"},\"breadcrumb\":{\"@id\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/when-ai-gets-it-right-most-of-the-time-the-problem-with-rare-cases\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/perit.ai\\\/blogs\\\/when-ai-gets-it-right-most-of-the-time-the-problem-with-rare-cases\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/when-ai-gets-it-right-most-of-the-time-the-problem-with-rare-cases\\\/#primaryimage\",\"url\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/wp-content\\\/uploads\\\/2026\\\/10\\\/When-AI-Gets-It-Right.webp\",\"contentUrl\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/wp-content\\\/uploads\\\/2026\\\/10\\\/When-AI-Gets-It-Right.webp\",\"width\":1672,\"height\":941},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/when-ai-gets-it-right-most-of-the-time-the-problem-with-rare-cases\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"When AI Gets It Right Most of the Time: The Problem With Rare Cases\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/#website\",\"url\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/\",\"name\":\"Perit Blog\",\"description\":\"Frontier training data, collected and graded by people who do the work.\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/#\\\/schema\\\/person\\\/954939b1549e8f3089b46850881d9775\",\"name\":\"Kshitij@Perit\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/cropped-Perit-Logo-150x150.png\",\"url\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/cropped-Perit-Logo-150x150.png\",\"contentUrl\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/cropped-Perit-Logo-150x150.png\",\"caption\":\"Kshitij@Perit\"},\"sameAs\":[\"https:\\\/\\\/perit.ai\\\/blogs\"],\"url\":\"https:\\\/\\\/perit.ai\\\/blogs\\\/author\\\/perit_ops\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"When AI Gets It Right Most of the Time: The Problem With Rare Cases - Perit Blog","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/perit.ai\/blogs\/when-ai-gets-it-right-most-of-the-time-the-problem-with-rare-cases\/","og_locale":"en_US","og_type":"article","og_title":"When AI Gets It Right Most of the Time: The Problem With Rare Cases - Perit Blog","og_description":"The Data Distribution Is Not the Real World A dataset does not become representative just because it is large. A model can be trained on millions of examples and still\u2026","og_url":"https:\/\/perit.ai\/blogs\/when-ai-gets-it-right-most-of-the-time-the-problem-with-rare-cases\/","og_site_name":"Perit Blog","article_published_time":"2026-10-03T09:12:17+00:00","og_image":[{"width":1672,"height":941,"url":"https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/10\/When-AI-Gets-It-Right.webp","type":"image\/webp"}],"author":"Kshitij@Perit","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Kshitij@Perit","Est. reading time":"8 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/perit.ai\/blogs\/when-ai-gets-it-right-most-of-the-time-the-problem-with-rare-cases\/#article","isPartOf":{"@id":"https:\/\/perit.ai\/blogs\/when-ai-gets-it-right-most-of-the-time-the-problem-with-rare-cases\/"},"author":{"name":"Kshitij@Perit","@id":"https:\/\/perit.ai\/blogs\/#\/schema\/person\/954939b1549e8f3089b46850881d9775"},"headline":"When AI Gets It Right Most of the Time: The Problem With Rare Cases","datePublished":"2026-10-03T09:12:17+00:00","mainEntityOfPage":{"@id":"https:\/\/perit.ai\/blogs\/when-ai-gets-it-right-most-of-the-time-the-problem-with-rare-cases\/"},"wordCount":1702,"commentCount":0,"image":{"@id":"https:\/\/perit.ai\/blogs\/when-ai-gets-it-right-most-of-the-time-the-problem-with-rare-cases\/#primaryimage"},"thumbnailUrl":"https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/10\/When-AI-Gets-It-Right.webp","articleSection":["Artificial Intelligence"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/perit.ai\/blogs\/when-ai-gets-it-right-most-of-the-time-the-problem-with-rare-cases\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/perit.ai\/blogs\/when-ai-gets-it-right-most-of-the-time-the-problem-with-rare-cases\/","url":"https:\/\/perit.ai\/blogs\/when-ai-gets-it-right-most-of-the-time-the-problem-with-rare-cases\/","name":"When AI Gets It Right Most of the Time: The Problem With Rare Cases - Perit Blog","isPartOf":{"@id":"https:\/\/perit.ai\/blogs\/#website"},"primaryImageOfPage":{"@id":"https:\/\/perit.ai\/blogs\/when-ai-gets-it-right-most-of-the-time-the-problem-with-rare-cases\/#primaryimage"},"image":{"@id":"https:\/\/perit.ai\/blogs\/when-ai-gets-it-right-most-of-the-time-the-problem-with-rare-cases\/#primaryimage"},"thumbnailUrl":"https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/10\/When-AI-Gets-It-Right.webp","datePublished":"2026-10-03T09:12:17+00:00","author":{"@id":"https:\/\/perit.ai\/blogs\/#\/schema\/person\/954939b1549e8f3089b46850881d9775"},"breadcrumb":{"@id":"https:\/\/perit.ai\/blogs\/when-ai-gets-it-right-most-of-the-time-the-problem-with-rare-cases\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/perit.ai\/blogs\/when-ai-gets-it-right-most-of-the-time-the-problem-with-rare-cases\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/perit.ai\/blogs\/when-ai-gets-it-right-most-of-the-time-the-problem-with-rare-cases\/#primaryimage","url":"https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/10\/When-AI-Gets-It-Right.webp","contentUrl":"https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/10\/When-AI-Gets-It-Right.webp","width":1672,"height":941},{"@type":"BreadcrumbList","@id":"https:\/\/perit.ai\/blogs\/when-ai-gets-it-right-most-of-the-time-the-problem-with-rare-cases\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/perit.ai\/blogs\/"},{"@type":"ListItem","position":2,"name":"When AI Gets It Right Most of the Time: The Problem With Rare Cases"}]},{"@type":"WebSite","@id":"https:\/\/perit.ai\/blogs\/#website","url":"https:\/\/perit.ai\/blogs\/","name":"Perit Blog","description":"Frontier training data, collected and graded by people who do the work.","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/perit.ai\/blogs\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/perit.ai\/blogs\/#\/schema\/person\/954939b1549e8f3089b46850881d9775","name":"Kshitij@Perit","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/09\/cropped-Perit-Logo-150x150.png","url":"https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/09\/cropped-Perit-Logo-150x150.png","contentUrl":"https:\/\/perit.ai\/blogs\/wp-content\/uploads\/2026\/09\/cropped-Perit-Logo-150x150.png","caption":"Kshitij@Perit"},"sameAs":["https:\/\/perit.ai\/blogs"],"url":"https:\/\/perit.ai\/blogs\/author\/perit_ops\/"}]}},"_links":{"self":[{"href":"https:\/\/perit.ai\/blogs\/wp-json\/wp\/v2\/posts\/950","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/perit.ai\/blogs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/perit.ai\/blogs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/perit.ai\/blogs\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/perit.ai\/blogs\/wp-json\/wp\/v2\/comments?post=950"}],"version-history":[{"count":2,"href":"https:\/\/perit.ai\/blogs\/wp-json\/wp\/v2\/posts\/950\/revisions"}],"predecessor-version":[{"id":952,"href":"https:\/\/perit.ai\/blogs\/wp-json\/wp\/v2\/posts\/950\/revisions\/952"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/perit.ai\/blogs\/wp-json\/wp\/v2\/media\/953"}],"wp:attachment":[{"href":"https:\/\/perit.ai\/blogs\/wp-json\/wp\/v2\/media?parent=950"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/perit.ai\/blogs\/wp-json\/wp\/v2\/categories?post=950"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/perit.ai\/blogs\/wp-json\/wp\/v2\/tags?post=950"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}