Aubergine was discussing the problem of AI modeling not being based on the best of the best.

Stephen Simmons
@simmonsactual
Mar 22
Model Collapse and You.

Here’s what’s on my mind right now about the trajectory of AI. While this may be old news for some, we need to be collectively aware of risk as we continue to integrate AI into everything we do.

We are in an era of extraordinary AI scaling…models that power everything from personalized recommendations and content generation to logistics optimization, search, and decision support. But a subtle challenge is accelerating beneath the surface, and it has the potential to undermine the progress we’re counting on.

The core issue: AI is eating its own tail (the Ouroboros effect).

Today’s large language models, and recommendation systems, were trained on mostly human-generated internet data (pre-2023). But generative AI now floods the web with trillions of words, images, code snippets, reviews, and articles—much of it is low-quality, hallucinated, or homogenized. When the next generation of AI models scrape the web for training data, they ingest this “faulty” synthetic content.

The Result? Performance collapses:

– Variation drops (this is not DEI…the AI models output bland, repetitive junk).
– Errors compound (hallucinations become the norm).
– Real-world accuracy plummets.

It’s classic garbage-in/garbage-out on steroids. Research shows an even smaller fraction of synthetic data can break scaling laws, resulting in what I can only describe as the ultimate self-licking ice cream cone.

This creates a self-reinforcing feedback cycle…AI validating itself through AI. Small hallucinations, biases, simplifications, and homogenized patterns compound recursively. Variety in outputs narrows dramatically. Errors amplify. Performance degrades in subtle but cumulative ways resulting in recommendations becoming less useful, creativity erosion, accuracy drifts, and reliability erosion.

Researchers have documented this as “model collapse,” where the system breaks the very scaling laws that have driven progress, leading to cascading mediocrity rather than a sudden failure.

The stakes are high…and this challenge will impact big tech especially hard. The good news is that this isn’t an unsolvable problem. There are some easy ways to solve:

– Deliberately anchor machine learning/training with human-informed data — verified, diverse (not DEI – literally differing perspectives), and grounded in real-world human experience, judgment, and variety.
– Implement rigorous provenance tracking to know exactly what fraction of training data is synthetic vs. human-originated.
– Enforce minimum ratios of human-generated content in every major retraining cycle.
– Prioritize ongoing curation, external verification, and human oversight to prevent recursive degradation.

These aren’t heavy-handed restrictions on innovation…they are disciplined practices that preserve AI usefulness, trustworthiness, and long-term advancement. When we protect the human input, we ensure these AI models continue to improve rather than echo their own mediocrity.Mar 22, 2026 · 1:03 AM UTC