The Corpus for Idiolectal Research (CIDRE)

1. Abstract

It is well known that the idiolect (the language of an individual) evolves over time. However, there is a lack of quantitative studies on this topic, due to the lack of large corpora (but see Barlow 2013; Mollin 2009; Petré et al. 2019 for a few examples). To study what is specific in an idiolect and how it evolves over a lifetime, we assembled, cleaned and dated the fiction works of 11 very prolific 19th and early 20th century French writers. This resulted in the CIDRE corpus counting 37 million words and over 400 books.

2. Motivation

We want to assemble a longitudinal corpus for stylistics studies, that is:

• Homogeneous (only fiction work)
• Open (contain only works in the public domain)
• Dense covering the 19th and early 20th century with overlaps between the different authors

3. Corpus Assembling

Criteria to select relevant authors:

• Large oeuvre of fiction novels
• E-books available as high quality e-pub files
• No collaborative works (e.g. A. Dumas)

Programming Scripts:

• Automatic download of e-pub files from open source library projects (e.g. Wikisource, etc.)
• Automatic cleaning of e-pub files (remove forewords, image captions, etc.)
• Manual correction

4. Data

5. Dating of Works

Important contribution of ours : Annotation of Books with year of writing

• Crucial for a diachronical study of the idiolect
• Might differ largely from:
  • First year of publication
  • Year of printing
• Provided in a metadata file with the corpus

Table 1: Some examples from the Gréville Corpus

6. Dates of Works

7. Availability and Licenses

• Download: https://github.com/oseminck/cidre/tree/v2.0
• Texts: public domain
• Metadata: Creative Commons - Attribution-ShareAlike 4.0
• Processing scripts: GPLv3 License

8. Acknowledgements

This work was funded in part by the French government under the management of the Agence Nationale de la Recherche as part of the "Investissements d’avenir" program, reference ANR-19-P3IA-0001 (PRAIRIE 3IA Institute).