Creating an artificial language for mitigating the Internet Information Retrieval recall problem

1. Introduction

Artificial languages (AL) come in many flavours and have been created for many purposes, consider for instance (Sanders 2016: 2) and are known at least since the 11th century (Higley 2007: 4-7). We create one in order to mitigate the recall problem of Internet Information Retrieval for Low Resource Languages (LRL). This problem refers to the fact that for any LRL, which one seeks to build a corpus for, compare (Scannell 2007: 4), one does not know how much of all available content on the web in the LRL is covered by what one has found. We created a corpus in an AL calling it Jucearm which we then posted in various places on the web. The AL we created is a so-called a posteriori language meaning that we took an existing language and altered it before applying a non-trivial and non-traceable letter substitution schema. We then place the contents from our corpus in bits in various social networks and on homepages. 5 students have then been provided with a noisy wordlist of Jucearm imitating a realistic situation for many LRL retrieval attempts and a short grammatical sketch and were told to search all available content on the web. We release the corpus alongside this publication. ¹

2. Creating Jucearm

We chose an a posteriori approach because the statistics of natural language are already inherent in the basis. And another advantage is that there are many resources and corpora for a widely used language on which to build the language. Our approach was to look for a major language that exhibits largely average statistical properties at various linguistic levels. Therefore, we considered WALS² for large web languages³ in the categories: Consonant Inventories (Maddieson 2013a), VowelQuality Inventories (Maddieson 2013c) and Syllable structure (Maddieson 2013b).⁴

We chose Spanish. We downloaded and merged all Spanish corpora from Universal Dependencies 2.6.⁵ Before substituting letters, we applied the following changes:

1. We substituted the upside-down question mark characteristic for Spanish orthography by a question particle, which we called ’hal’ as its Arabic equivalent. However, this was applied to content and yes/no questions.
2. We substituted all feminine definite articles in either singular or plural by their masculine counterpart. Although this transformation did not change the complete gender system, it is statistically significant at least in theory.
3. We substituted the synthetic future with the -ra type endings by an analytic one with the verb volver. Thus ’tu comerás’ becomes ’tu vuelves comer’.
4. Finally, we added a peculiar feature to the source language, which is not very widespread in European languages, at least not for the expression of grammatical properties: partial reduplication. For superlatives of Adjectives or Adverbs, which were continuous tokens as ’el más X’, we substituted by the initial sequence until the (first or if present) second vowel, which resulted in a full reduplication if the token was short, such as bajobajo. Examples: descodesconocido, impoimportante, evievidentes.

After this, we applied a letter mapping with a sum total of 36 transformations, including three 2:1 and three 1:2 transformations. Spanish is thus not trivially reconstructible. If one would expect text in the AL to be encrypted, then a letter frequency analysis independent of the letters could reconstruct it otherwise.

Upon development, we created more than one mapping and each time consulted the statistics of the generated text until we found the result to be normal with respect to statistics. The tools we used to analyze the generated texts were:

1. Analysis of the distribution of letters with R in comparison to a variety of languages.
2. Zipfplots created with R and compared to Zipfplots of other languages (Montemurro 2011).
3. We combined usually three words (low frequency, mid-frequency and high-frequency words) and noted the primary languages of the first 20 pages found in a web query. The desired result should i) be diverse and ii) not feature a lot of Spanish.

In the grapheme system of the AL appear grapheme sequences which seem difficult to pronounce and unusual, such as <cj> or <sj>. However, these grapheme sequences constitute graphemic units as parts of writing systems and could for instance represent palatalized sounds in our AL. For example, <cj> is a part of the Friulian writing system and represents the voiceless palatal stop /c/ (Miotti 2002) or <sj> which consitutes the voiceless velopalatal fricative /ɧ/ in Swedish (Holmes / Hinchliffe 2013). This means that the language is readable applying some idiosyncratic graphemic unit to phoneme rules such as <sj> /ʃ/ as many languages do possess. An example of the original Spanish and our correspondent a posteriori AL:

Spanish: Es como si los hubieran estado golpeando contra el pared durante horas.

Jucearm: Uz sunu zo sjuz figoamez azpebu dusjwaezbu suzpme asj wemab bimezpa fumez.

3. Named Entities

Transforming a text in the above-described way would also affect Named Entities(NE), which can be orthographically much more complex than average words of a language and often include abbreviations/acronymes. A text without Nes would be unusual and a text with almost only language internal NEs and no loaned ones may be even more so.

We extract a list of NEs using spaCy’s Named Entity Recognizer (NER).⁶The data-set we used for the extraction consists of Wikipedia dumps of 14 different languages: Afrikaans, Albanian, Basque, Danish, Estonian, French, Italian,Croatian, Lithuanian, Polish, Portuguese, Romanian, Swedish and Turkish. We then used the program wiki-extractor⁷to extract only text data from the Wikidump. Using spaCy and individually trained models, we extracted overall 8.000 tokens of Named Entities.

We then substituted all the NE tagged instances from the UD base corpus by one random entry of the NER list. Sometimes we slightly adapted the names to typical Jucearm letter sequences. We classified the result, the final corpus, with Facebooks fastText ⁸which gave Slovenian as a result.

4. Migration into the internet

The (main) goal of this paper is to imitate/emulate a situation of a typical LRL. We use our experience in the work with LRLs on the web (Hoenen et al. 2020) to find suitable places. Yet, we want to be legally certain to put our AL in specific places on the internet, where it is first legal/allowed and second does no damage. In order to verify the validity of both mentioned aspects, we looked at the general terms and conditions as well at all privacy policies on every page in which we have placed our text. Furthermore we made sure, that it is possible to delete each placed text after the experiment. We placed some content on our homepages (2 domains, with a pdf also placed), placed some content on social media and bid a ‘book in Artificial Language’ with an excerpt of Jucearm on a vendor platform. We also placed a genuine sentence in the AL on Wikipedia in a fitting article where it persisted in Ukrainian.

5. Live-Experiment

We created a “starter kit” in the An Crúbadán format⁹with a word list and a short grammatical sketch. Based on the starter package, we instructed 5 participants to search for the placed contents in the AL. The results of the experiment showed that all participants had found at least one of the subpages on our own websites; four had also found more than one piece of content. Four participants found the text in the Wikipedia article and three found our classified ad in the AL. In the end, none of the participants found the entries on Social Media. All participants used search engines (primarily Google) for the experiment, by using queries of multiple terms, single words, or the name of the AL itself.

6. Discussion and Conclusion

Since most probably for no living medium-size LRL on the web all places of content are known with certainty, approaching the quality of the simulation must remain controversal. Despite being only a small start, the results are however encouraging and could point to modern corpus crawling for LRLs being quite effective concerning recall.