AI, hallucinations, and a suddenly intensified warning narrative
I came across this topic because, since the beginning of November 2025, I have encountered a strikingly consistent narrative: across various German-language media formats, the phrase “ChatGPT and similar systems should never be used like a search engine” has suddenly appeared with great frequency. This is not a single column, but a short series of articles that are very similar in their choice of words and framing of the problem.
One example is the article by RuhrkanalNews from November 11, 2025, entitled “ChatGPT is NOT a search engine.” It explicitly warns against using ChatGPT as a replacement for traditional search engines. The editorial team describes the system as treacherous, as it allegedly packages nonsense in well-formulated sentences and is therefore not a research tool and cannot replace journalism, science, or independent source verification.
A second reference point is the campaign by Mimikama. Since the beginning of November, the organization has been distributing a visually striking tile with the message “ChatGPT is NOT a search engine,” combined with the claim that a large proportion of users are using the system as if it were a replacement for Google and other search services, which is considered dangerous. In a follow-up text, Mimikama comments that the post has triggered significantly more reactions than expected, which shows that this narrative is widely perceived.
A third building block is the interview with computer scientist Katharina Zweig, which was published in the Frankfurter Allgemeine Zeitung and is circulating on social media. Not only does it warn of a major AI bubble, but it literally elevates not using ChatGPT as a search engine to the status of a first rule. The reasoning behind this is that language models do not have a real knowledge database, but merely string words together.
These three examples come from different contexts: a local news portal, a fact-checking platform, and a major national newspaper. In terms of content, however, they all say almost the same thing: ChatGPT is not a search engine, language models are structurally unreliable, and the average user is overwhelmed by this technology. In addition, parallel articles appear in publications such as the Austrian Standard, which describe AI browsers as a nightmare for security and privacy, thus extending the warning to include a security policy dimension.
For me, this paints a picture of a warning narrative that intensified in November, which can be summarized as follows: AI is tempting but dangerous and should not be used for research or everyday decisions, at least not by the general public.
Anthropology instead of technological determinism
My own observation starts at a different point. Before I talk about technology, I first think about people. To live consciously in this world, it is not enough to be able to retrieve information. It is crucial that I realize what I know, what I do not know, and that there is a realm of unknown unknowns. In other words, questions and problems that I have not even formulated because I lack the necessary perspective.
This epistemic triangle is independent of the technology used. It applies to classic search engines, books, conversations with experts, and also to the use of AI systems. Those who do not take stock of their own limits of knowledge and ignorance will make poor decisions even with the perfect search engine. In this sense, the central dimension is not technical, but anthropological.
I have been working intensively with AI systems since October 2022. I had my first encounters with AI, including as legal evidence systems, between 2014 and 2017. During this time, the technical basis has developed significantly. I have seen models that are based exclusively on pre-trained world knowledge, and today I use versions that can search the internet in real time. It is crucial to my way of working that I consistently work with explicit requirements. I demand source references for all relevant factual claims, I prohibit the invention of references, and I demand verification from several independent sources for controversial or time-sensitive topics.
At the same time, I use my ability to store context and long-term memories. If an error is identified, such as an inaccurate date or a false attribution, I expect this error not to be repeated in the future. In addition, I take the time to formulate my questions precisely. I try to treat my prompt as I would formulate a scientific question. This means that I first clarify the subject matter, the relevant time period, the normative framework, and the terms used.
Under these conditions, I observe two things. First, the hallucination rate has decreased significantly. Errors still occur, but they are rare and usually identifiable. Second, the nature of the errors is shifting. What used to often culminate in fictitious books, judgments, file numbers, or article titles now tends to manifest itself in ambiguities at the margins, such as in borderline cases involving dates or very specific, poorly documented detailed questions.
In my view, this shows that although the technology has its limits, these limits depend to a large extent on how reflectively I use the system. Those who consciously ask questions, demand sources, and keep their own knowledge limits in mind produce a completely different error structure than those who treat a single, unquestioned answer as the final truth.
Systemic level: gatekeepers and narrative protection
When I shift from the individual to the systemic perspective, another question arises. Namely, in which larger field of discourse these warning articles are located. For several years now, disinformation has been described in German-speaking countries as one of the greatest dangers to democratic societies. Studies by media associations and foundations have concluded that a large part of the population perceives misinformation as a threat to democracy and social cohesion.
At the same time, traditional media, publicly funded institutions, and various non-governmental organizations have positioned themselves as gatekeepers of facts. They define which sources are considered trustworthy and which positions are classified as disinformation. In this setting, AI systems initially appear to be an additional disruptive factor. In theory, they can reveal alternative sources, formulate counterarguments to established narratives, and juxtapose different perspectives.
The November articles fit relatively seamlessly into this existing framework. They do not emphasize that AI is a tool that produces very different results depending on how it is used. Instead, the focus is on risks: hallucinations, disinformation, and an alleged structural overload for the average user. AI is thus not primarily understood as an extension of cognitive capacities, but as a potential amplifier of misconceptions.
There is no central master plan in this picture. Systems are not controlled by a single instance, but by incentive structures, institutional logics, and shared interpretations of problems. If the key question is how to stem a supposed flood of disinformation, then it makes systemic sense to problematize tools that could strengthen access to information and argumentation skills outside of traditional gatekeeper channels.
In this logic, the narrative that ChatGPT is not a search engine and not a reliable source of information becomes an instrument of narrative protection. It stabilizes the position of those actors who see themselves as verified fact providers, while at the same time discrediting the spontaneous, non-institutional use of AI as a research aid.
Where my thesis holds water and where it remains speculative
I think it is important to make a clear distinction between what can be deduced from the observations and where I am moving into the realm of plausible but unprovable interpretation.
Firstly, the temporal concentration is well documented. In the fall of 2025, there are a number of articles that convey the same leitmotif in quick succession. ChatGPT is not a search engine, its use as such is dangerous, and AI systems are unreliable as a source of information. Several examples come from different types of media, but some use identical wording.
Secondly, the embedding in an already established discourse in which disinformation is described as a major structural threat to democracy and social stability is well documented. In this discourse, it is consistent to view new technical systems primarily from the perspective of risk management.
The assumption that a powerful AI tool challenges the interpretive authority of traditional gatekeepers is also easy to understand. Those who are able to use a system to compare different sources and reconstruct opposing positions are less dependent on the curated, visible selection offered to them in traditional news programs, for example. This effect follows from the structure of the tool and does not require a conspiracy theory.
It remains speculative whether individual actors deliberately aim to discredit AI as a research tool in order to protect their own narratives. Public texts do not provide any explicit information on this. It is entirely possible that conviction, professional identity, and institutional logic are sufficient to produce the rhetoric observed, without there being a coordinated strategy behind it. It also remains unclear to what extent warnings about the misuse of AI are covered by real cases of abuse and where they are more likely to represent anticipatory fear projects.
In other words, my thesis holds water where it describes the structural fit between a more control-oriented discourse on disinformation and the problematizing portrayal of AI. It becomes speculative where one might conclude from this fit that individual institutions have specific control intentions.
“Hallucination experience”
A central point of reference for my assessment is my own experience with hallucinations when dealing with AI systems. Early model generations had a significantly higher tendency to generate plausible but fictitious answers when knowledge was lacking. This mainly affected very specific detailed questions, poorly documented facts, and fictional combinations of real and invented elements.
With the possibility of real-time research and the introduction of clear guidelines prohibiting the invention of sources, this pattern has changed noticeably. Errors still occur, but they are more often in the form of inaccuracies or incompleteness rather than completely invented structures. In addition, identified errors can be avoided in the future through appropriate use of the memory component.
Of course, this observation cannot be generalized. It is the result of a specific approach. I work with detailed prompts, I require source references, I question statements, and I initially regard every answer as a hypothesis, not as definitive truth. Under these conditions, the hallucination rate is extremely low. In this respect, the blanket warning against hallucinations as a structural feature falls short. It ignores the fact that the error structure of a model depends not only on its architecture, but just as much on the way people use the model.
At the same time, it would be dishonest to downplay the risks. Anyone who works without source criticism, who accepts AI answers at face value without classifying them in their own knowledge base, can certainly be misled by a few serious errors. Media outlets that emphasize this danger have a valid point in this respect. But they only tell part of the story. The other part is that, when used thoughtfully, the same technology can be a very powerful extension of intellectual capacity.
For me, this leads to a simple conclusion. The real protection against undesirable developments lies not in the blanket discrediting of the tool, but in the strengthening of epistemic competencies. This means that people must learn to recognize the limits of their own knowledge, ask precise questions, check sources, and tolerate contradictions. Those who do not do so are vulnerable even without AI. Those who do can use AI productively.