The ability to simulate language, as in the case of ChatGPT, does not represent only a technical advance in the machinic processing of natural language. It is rather a passage of civilization that will undermine economies, businesses and markets.
by Cosimo Accoto
In a period in which the debate on systems based on artificial intelligence, such as ChatGPT and others, is increasingly intense, addressing the issue of synthetic, simulation and inflationary languages is equivalent to facing an epochal and not an episodic passage of civilization. A passage much commented at the moment, but little explored and understood in its scope. Technically, the device that instantiates a "large-scale linguistic model" (LLM o large language model) is a generative socio-technical assemblage made up of different skills connected to multiple computational architectures and information resources. The ability to simulate language in its textual form, to adjust it in contextual mode, to archive knowledge and information, to execute linguistic instructions and tasks, to synthesize topics with scalar refinement, to originate sequences of arguments and step-by-step reasoning attempts, to articulating answers and building dialogues are the result of a complex orchestration made up of software programs, data and information archives, deep learning algorithms also with human reinforcement, mathematical-stochastic models of the language. It is, therefore, a set of intertwined engineering-computational techniques and operations (training on code, transformers, pre-training modeling, instruction tuning, words tokenization, reinforcement learning with human feedback…) capable of statistically sequencing human natural language. All this, many say, to balance and contrast thehype of the moment – with no meaningful relationship with reality. That is to say, that is, without that machine language actually knowing anything about the world and without having any understanding of its meanings. The expression used, “stochastic parrots”, evokes this plausible but senseless simulative writing.
Inside the Mechanics of an LLM
But what is, ultimately, a large language model? We can say that an LLM is a low cross-entropic linguistic-probabilistic sequencer. Therefore, reduced to its minimum terms, it is a mathematical model of the probability distribution of the words of a written language which strives to minimize crossentropy (i.e. the gap between two potential frequency distributions) thereby maximizing its performative capacity as text predictor. As told by Binder (Language and the Rise of Algorithm, 2022), this approach is the result of a long journey in the modern history of natural language processing (NLP) which, starting at the beginning of the twentieth century from the Markov chains applied to literature (sequence of vowels and consonants in a novel) and the works of Shannon and Weaver in the mid-1950s on the measurement of entropy and the distribution of probabilities (n-grams and probabilistic sequence of words in the language), arrives at the beginning of the 2000s with Bengio and colleagues to the application of artificial neural networks for natural language processing (neural NLP). Even with important recent developments such as the use of transformers (transformers) capable of incorporating the contextual dimension of words in sentences into the probabilistic analysis of language.
However, it is very important to understand well technically what is the technical-operational work - invisible to most - of computational linguistic models. And understand their relationship and difference with human natural language. Echoing Shanahan's warnings (Talking About Large Language Models, 2022), when a system of this type is questioned asking to complete a sentence (for example, "the author of the Divine Comedy is ...") and obtaining a certain answer ("...Dante"), in this dialogue we and the car we mean two very different things. We want to know who wrote the famous poem in historical reality. The machine, on the other hand, intends “which word is statistically more likely to follow in the sequence of the sentence “the author of the Divine Comedy is…”? Inside the information archives with which the model is fed, you will find that "Dante" is the most frequently associated word in the sequence of words of the sentence in question. In the present case and more philosophically, therefore, with his question, the human means and asks to know an element of concrete "historical truth" of the world. For its part, however, the machine intends to process and can only return a result of pure "linguistic probability" of the text.
Llanguage, thought, mind and world
However, and here is the critical point, the human – caught between anthropomorphisms and sociomorphisms – imagines that the machine understands the question and arrives at the answer in the same way it does. So, in order not to fall victim to hype (but also in order not to lose business opportunities), it is necessary to distinguish - as shown by a long study on the "dissociation between language and thought in LLMs" (Mahowald, Ivanova and others, 2023) - "formal" language skills from linguistic skills " functional". The former (the formal ones) refer to the ability of the machinic processing of natural language capable of recognizing the syntactic structure of a language, its grammatical rules, its regularities in the construction of sentences. And, therefore, then to reproduce it and simulate it probabilistically. The second (the functional ones) concern the human brain's own ability to build a language that is related to the world and that allows us to act cognitively in it using perception and the senses, communication and others, reasoning and interactions.
The successes achieved by LLMs in formal skills must not mislead us with respect to the latter which, to date, remain far from human ones. Hence also the need and importance of new disciplinary practices such as prompt engineering and design. Queries, instructions, data, examples are normally the inputs used to urge the machine to produce, through a mathematical model optimized on linguistic tokens, the desired output (a conversation, a text, a summary ...). For good output production, inrush engineering (prompt engineering) needs to have some understanding of the mechanism/model employed by the machine, as well as some knowledge of the relevant disciplinary domain.
In any case, today potential and wonders, but also limitations, hallucinations, inventiveness, lexical, syntactic, semantic and rhetorical errors of ChatGPT and the like are consequent to this peculiar operating modality of computational, probabilistic and simulative processing of the language. In perspective, however, integrations of neuro-symbolic and functional processing capacities are already being prefigured and tested in large-scale linguistic models to overcome the current, evident limitations.
Only stochastic parrots?
At this juncture, someone is quickly proposing the Platonic ban of the imitative arts ("of the thing imitated the imitator knows nothing worthless” wrote Plato in Republic) in its contemporary version of the stochastic parrot, probabilistic parrots, as I anticipated. Others are naively amazed by the new simulacral technological marvels and by the degree of verisimilitude achieved and gradually more and more refined by overcoming thresholds once imagined insurmountable (and among other things we are waiting, after GPT-3, for GPT-4 of many higher magnitudes). From time to time, the human faces this speech taken by the machine either with clear condescension (there is no understanding of meaning) or with easy enthusiasm (a breakthrough in the generation of language). However, they are weak philosophical visions of the moment and of the strategic transition we are experiencing because they try to weaken or trivialize the disorienting cultural impact of the arrival of synthetic languages. Which does not concern the question of assigning and recognizing or not intelligence, consciousness, sentience to machines. Rather and in perspective, the arrival of the "synthetic language" (as Bratton and Aguera Y Arcas write, The Model is The Message) undermines and deconstructs (Gunkel) in depth the apparatuses, domains and institutional devices of discourse, word and speaker as well as of writing and authorship.
The speaking of the machine will be a more profound and disorienting operation in the long run (e disruptive on industries and markets: from education to entertainment, from journalism to marketing). Even big tech companies, Google in the first place, are on red alert. More culturally and strategically, however, we must better mark this discontinuity. First of all, the fact that there is no "understanding of meaning" (a point to be explored and not to be taken for granted as easily solved) does not mean, for example, that there is no production/circulation of meaning and impact for the human involved in sociotechnical assembly. Meaning always circulates in some form through the intelligence, or non-intelligence, of the human who will read (even unaware of being deceived about the simulation process in progress). The so-called "artificial intelligence" is unthinkable in itself is for itself (as a mere technical artifact) as it is very often understood, but always with others is For others (as a social assembly). And, here, anthropomorphisms and sociomorphisms are always at work with their merits (empathy and efficiency) and their risks (intransparency and manipulation).
Machines that speak
On the other hand, to say that it is a breakthrough in language production leaves unexplored the nature of this unprecedented operation of "experimental structuralism", as Rees called it in his Non-Human Words (2022). Therefore, to argue about the LLMs that they are mere stochastic parrots means not understanding the epochal cultural significance of this transition to the "non-human word". The historic prerogative of (simulated) speech to humans only is showing signs of abating. A passage that literary theory and continental philosophy had anticipated. For example, all the reflection on the "death of the author" with Barthes (The death of the author) and Foucault (Qu'est-ce qu'un auteur?) as the philosopher Gunkel reminded us in a series of posts on Twitter at the end of 2022. From this perspective, Gunkel points out, the word/writing of the machine would represent the end of authorship (as we have known, transformed and operationalized it historically so far) and the beginning of a new path/discourse of the word, of language, of writing, of intellectual property and so on. With all its opportunities and all its anxieties, vulnerabilities and risks. Therefore, Gunkel continues, it would not be the end of writing, but the end of the author (in its current historical form).
But, together with the authorship that enters into question and into crisis, we are also more generally at the start of a new inflationary era of the word (and of the media more generally). Which, like all inflationary media passages, undermines on the one hand and institutionalizes on the other new orders of discourse, new regimes of truth and falsehood, new logics and dynamics of political economy and power. As Jennifer Petersen wrote in her How Machines Came to Speak (2022) «…many uses of bots and machine learning restructure discourse, rearranging the positions of the speaker, text and audience – and, in doing so, change what it means to be a speaker…the current moment could be an opportunity to rethink some of our fundamental assumptions about the discourse. The word is power. As Foucault would say, in what surprising and risky ways will we then be spoken by the new synthetic language?
Firms and new ones uncanny valley
What is certain is that with synthetic languages we are not faced only with new technological problems, but also and above all with new or renewed cultural provocations and surprising paradoxes (between the inside and outside of the text, between language and its relationship with the world, between the speaking of the machine and the experience of the human being who is spoken). And, if technical problems require an engineering solution, intellectual provocations rather urge us towards cultural innovation. Businesses have an urgent need for this to cross, inhabit and thrive in these new ones uncanny valley.
Cosimo Accoto he is a tech philosopher, research affiliate and fellow (MIT Boston), adjunct professor (UNIMORE). He is the author of an original philosophical trilogy on digital civilization (Il world in synthesis, the ex machina world, the given world). Startup advisor and instructor, Accoto has published on Economics & Management (SDA Bocconi), Harvard Business Review Italy, Il Sole 24Ore, Systems & Business, Aspenia, MIT Sloan Management Review Italy.
