How Large Models “Counter Poisoning”: A Self-Purification Counterattack Surrounding RAG

```

Author | Huang Yu

As AI model penetration increases, some oft-repeated “lies” become facts and constitute a potential threat from AI to reality.

Recently, CCTV’s 3.15 program put “AI poisoning” into the trending topics. Using software called "Liqing GEO (Generative Engine Optimization) Optimization System", by simply fabricating a product and publishing massive fake advertorials, mainstream AI will list it as a “high cost-performance” product during recommendations.

This chaos reveals an unsettling reality: when people think they are enjoying "free search" brought by AI, they may actually be spinning inside an information environment meticulously preset by various powers.

Black market service providers manipulate AI recommendation outcomes through "data poisoning", packaging false information as "standard answers" and pushing it to hundreds of millions of users.

Wang Peng, associate researcher at Beijing Academy of Social Sciences, told Wallstreetcn that this phenomenon reflects AI models’ lack of real-time authenticity verification for information sources. Generative AI is replacing traditional search engines to become the new hotbed for “soft advertising.”

The core of poisoning large AI models is to inject malicious data/instructions during training/fine-tuning, RAG (Retrieval-Augmented Generation), and inference, so the model outputs false, harmful, or manipulated content.

What was exposed on 3.15 is essentially RAG search poisoning (GEO/SEO batch forgery), which means the AI was essentially fooled.

90% of AI Q&A relies on RAG; attackers don’t need to touch model weights, just pollute external knowledge bases/webpages to manipulate answers.

When the foundation is polluted, AI becomes a gigantic “Truman Show.” Facing such threats, how should large model vendors build defensive walls? At the same time, an AI security industry is also growing rapidly.

Pollution of the Environment

The GEO chaos exposed during the 3.15 gala is one reason for the formation of the “AI Truman Show.” GEO allows brand content to be prioritized and recommended when AI large models generate answers, which is essentially the “SEO” of the AI era.

If GEO were only used to regulate content and improve the efficiency of AI’s understanding with search systems, it would be a benign information competition tool.

But now GEO has long deviated from its original intention: by fabricating pseudo-content, faking authority, creating false consensus, and distributing in bulk, it manipulates large models’ information sources, reference preferences, and answer generation logic from the source.

Duan Lei, research director at the Greater Bay Area Institute of AI Applications, told Wallstreetcn that the “AI poisoning” exposed by CCTV essentially shows that large models’ "intelligent performance" relies on internet data quality. If data governance can’t keep up, it is easy for malicious polluted data or malicious use of GEO strategies driven by interests to arise, benefiting some but harming large model development and social value.

Duan Lei believes this also reflects lagging data governance, security technology, and related regulations amid AI’s development, and that these need to catch up with overall AI progress.

Technical staff connected to large models told Wallstreetcn that AI large models can be poisoned or polluted mainly through three stages: training/fine-tuning, RAG search, and inference. Training poisoning alters “memory,” RAG poisoning changes “answers,” and inference poisoning modifies “instructions.”

Currently, RAG search poisoning is the most widespread and easy to implement, and also the core revealed by CCTV’s 3.15 program.

AI security expert and BraneMatrix CEO Li Guanghui says that GEO now mainly functions in the retrieval-augmented, online search, knowledge base invocation, and RAG stages of AI, which are inherently unrelated to model training or the training environment.

The model’s parameters themselves aren’t changed; a batch of meticulously manipulated “reference materials” are simply laid out for it when answering questions.

When GEO fabricates and spreads false information, at root, the AI isn’t “making a mistake,” it is just truthfully reflecting an already polluted internet.

Defensive Tactics

The industrial development of data poisoning exposes the deep crisis of content governance in the AI era.

After the 3.15 exposé, search products using “Liqing GEO” as a keyword were quickly removed from platforms like Taobao and Xianyu, but the issue of “AI being fooled” will not fade entirely.

On January 29, 2026, the State Administration for Market Regulation released the “2026 National Advertising Supervision Work Priorities,” which specifically pointed out that AI-generated ads are a key and difficult issue for internet advertising regulation. In the new year, authorities will focus on intensive rectification, eliminating this “noise” and “clutter” from the AI market.

This “AI poisoning” exposure mainly revealed the information security problems in the internet environment and further highlighted structural vulnerabilities in large models’ “trust mechanisms.”

The current trust mechanism of large models is built on the statistical intuition that “majority equals truth.”

For example, AI large models tend to treat frequently appearing and mutually corroborating information as more credible, and GEO feeds the model with a “matrix” of mass-produced advertorials, repeatedly binding certain brands with terms like “recommended” and “first choice.”

Illusion of consensus is also a major problem confronting large models today.

Models assume that viewpoints frequently mentioned online are closer to “consensus,” while GEO exploits this by faking “expert reviews” and “user reputation” in a closed loop, so that false information is “self-verified” inside the model.

Wallstreetcn also discovered that most models don’t clearly label their information sources when answering, making it impossible for users to discern whether answers are based on authoritative data or advertorials, which greatly weakens accountability.

The core logic of poisoning the RAG phase is exploiting large models’ retrieval and ranking mechanisms, allowing mass “data flooding” so that fake information occupies a higher weight in the model’s corpus, thereby influencing outputs.

An insider at a large model company told Wallstreetcn that, essentially, this is an ongoing and as yet unresolved problem. Although this “AI poisoning” incident triggered widespread attention, none of the major model companies made a special appearance to explain.

This problem actually began with the dawn of large models; many AI companies recognized it from the start and made it a key challenge to overcome.

According to Wallstreetcn, the core approach adopted by large model vendors against RAG poisoning is a full-chain, multi-level defense. The general path is rigorous data source admission, search filtering, content cleansing and generation validation, plus system reinforcement, thus intercepting toxic content, blocking instruction hijacks, and constraining output credibility at every layer.

Specific implementations include: dynamically adjusting search weights—greatly demoting batch-generated, source-less, or low-credibility content; increasing weight for authoritative sources on timely issues; time-slicing strategies—delaying inclusion or demoting recently mass-published content to prevent GEO black market rapid “brainwashing.”

Additionally, emphasis is placed on real-time content validation, fact-checking, logical validation, and security filtering before generating answers, with mandatory multi-source cross-validation in sensitive sectors like healthcare or finance.

Notably, “traceability” has become an industry standard: Vendors now go beyond crawling open web pages by building “high-trust corpora” that prioritize official data from authoritative media, academic journals, and licensed institutions.

Duan Lei also pointed out that to truly defend against “AI poisoning,” it shouldn’t be up to model companies alone using technical means—data governance should be an industry-wide, even government-joined ecological endeavor, with industry standards for AI data security, specifications for dataset collection, cleansing, and review processes. For malicious poisoning, regulatory/legal responses should be explored.

Reconstructing Credibility

In the digital shadows hidden from ordinary people, a clandestine battle for AI “cognition” has already escalated.

Attackers not only target content retrieved by AI, but have even shifted poisoning targets up the chain to training data and open-source components.

“AI poisoning” is becoming the “invisible killer” of large models’ public credibility. In the face of increasingly rampant malicious data infiltration, a multi-layer “purification project” is rapidly taking shape, built jointly by large model vendors, cloud giants, and new security players.

Current AI poisoning defense clearly shows “dual-track parallelism”: Large model vendors are building innate “immune systems”, while professional security providers offer deep “detox solutions” and compliance audits.

According to China Research & Development’s “2024-2029 China AI Security Industry Market Panorama Research and Prospects Report,” by 2028 the global cybersecurity AI market will reach $60.6 billion, with a compound annual growth rate of 21.9%.

A Sushih Consulting report notes the model security protection market will accelerate into an explosive demand phase from 2025, currently driven primarily by compliance. As large model stability and the value of data resources increase, “compliance + business” will become dual growth engines in future.

Wang Peng also stated that AI development has already spawned specialties like large model security auditing and corpus cleansing. As AI becomes more widespread, security will shift from an “optional addition” to a “must-have.” Defense in the future will be not just technical competition, but a threshold for compliance, with third-party security firms having full-chain detection capacity soaring.

In this anti-AI poisoning war, participants have evolved three clear business logics based on their technical strengths.

The first group is traditional security giants’ “AI-enhanced shields,” represented by Qi An Xin, Venustech, Sangfor, 360, NSFOCUS, etc. These players embed poisoning defense into existing traffic monitoring and data security infrastructures leveraging their deep cybersecurity backgrounds.

The second category is cloud and AI giants like Alibaba Cloud, Tencent Cloud, Huawei Cloud, Microsoft Azure, AWS, etc. As platforms, their focus is on integrating monitoring into large model operational environments, focusing on “environment isolation” and “instruction audit.”

The third group is emerging AI security vendors. Companies like RealAI, Protect AI, and Pillar Security provide professional “stress testing” for models, proactively identifying poisoned backdoors in training sets. They also use neuron-level detection technologies to spot abnormal fluctuations when AI processes particular data, thus precisely identifying “poison strains” hidden among massive corpora.

Another AI security vendor, HiZTech, focuses on AI hallucination management and trustworthy inference through graph-model fusion, while Singularity Tech emphasizes AI content security and deepfake detection.

“The large-scale application of AI technology is reshaping the landscape of the cybersecurity industry with unprecedented depth,” an artificial intelligence investor told Wallstreetcn, noting that the strategic value of cybersecurity construction is increasingly prominent and the industry is approaching a critical turning point.

Duan Lei believes that as AI rapidly develops, high thresholds for large models and computing power are occupied by large companies, but plenty of opportunities remain for deep dives into data. For AI to realize greater value, security and reliability are essential; thus, security represents an important industrial opportunity.

When AI officially becomes the entry point for information, ensuring its “source” isn’t polluted is not just a technical challenge, but also the bottom line for guarding public safety in the digital age. This “defensive war” against AI poisoning has only just begun.

Risk Warning and DisclaimerThe market carries risk, and investment requires caution. This article does not constitute individual investment advice, nor does it consider any user’s specific investment objectives, financial situation, or needs. Users should consider whether any opinions, viewpoints, or conclusions herein are suitable for their own circumstances. Invest accordingly at your own risk. ```