EGRAPHSEN. Possibilities and Perspectives of the Digital Painter Attribution for Attic Vases. Early Red-Figured Cup Painters: Douris – Makron and the Brygos Painter

Within the field of Ancient Studies, the painted pottery produced in ancient Athens has always played a prominent role. During the 5th century BC, when Athens became the strongest power in the Greek world, pottery workshops flourished, and the pictures painted on various types of pots during this period offer insights into the complex social, economic, and political frameworks of this ancient society. Until today, Attic painted pottery remains a key strand of archaeological research. One of the most important contributions to this is the work of Sir John Beazley (1885–1970) who refined a method of identifying individual painters by their style, similar to handwriting recognition (Beazley 1922): each painter presumably had his own way to place strokes within minor details of the painting, particularly when defining the folds of a figure’s clothing, nose, collarbones, ears, and so on. Only a tiny fraction of the extant 50,000 Attic vases painted in red-figure technique (i.e., a technique used between 530 and 400 BC by means of which the figures retain the red color of the pot’s clay, while the background and strokes are painted with a clay slip that turns black during the firing process) was indeed signed with the painter’s name. However, Beazley’s method enabled him to reconstruct a system of interacting and collaborating potters and painters, both known and unknown by inscriptions. His painter attributions remain standard reference for archaeologists today.

Nonetheless, over the last decades, Beazley’s method has repeatedly been criticized as connoisseurship and as not being based on purely objective criteria (Neer 1997; Graepler 2016). This is an issue that we address in our project by computer-assisted analysis of Attic vases and their paintings. By means of pattern recognition via a deep Convolutional Neuronal Network (CNN) we are presently testing and re-evaluating one of the most important yet controversial methods of classical archaeology based on systematized attribution criteria.

By training the CNN based on Beazley’s attributions, we aim at (1) testing how and to what degree we can simulate a human-expert attribution of Attic vase paintings, and (2) comparing the results of the "human expert" Beazley, the trained CNN, and the attributions of other researchers to discuss the potential and the limitations of painter attribution via machine learning and via traditional methods still used by archaeologists today respectively.

In regards to the data set, we are focusing on red-figured Attic vases from the 5th century BC. In a first step, we chose the painters Makron and Douris (both known from inscriptions) and the so called Brygos Painter (no signature), since they were active during approx. the same period (ca. 490–470 BC), preferred the same vase shapes (mostly kylikes = cups), and display a similar range of topics and figures on their vase paintings. In a second step, we expanded the range of vase painters to a number of ten, covering the full range of the 5th century BC. All details of the respective figures including body parts and clothes, objects, and ornaments depicted were annotated by a team of six people, all trained in classical archaeology. We annotated approx. 70 images per painter with almost 5000 object annotations in total, in order to provide training sets of nearly equal size.

The fragmentary state of many vases, the often low quality of the photos available, and the distortions of the figures due to the curvature of the vessel’s shape raise several questions regarding the selection of the data set, all of which will be addressed in more detail within our paper. The greatest challenge, however, is the extraordinary complexity, heterogeneity, and variety of the paintings. Since any detail may contribute to the identification of a painter, the number of classes annotated was much higher than the number of paintings that were attributed by Beazley to each single painter; hence, the number of classes annotated per painting in relation to the number of paintings in general is disproportionally large. Additionally, some classes appear in only a few paintings. This led to several developments and adjustments of the CNN model.

First, we pretrained MobileNetV3 (Howard et al. 2019) on the much larger ILSVRC-2012 data set (Deng et al. 2009) to embed the images in a latent space. Then, a three-layer feedforward Neural Network with 64 neurons per layer and ReLU activation functions was trained for full image classification using several data augmentation techniques due to the low-data regime. This alone enabled us to identify paintings signed by (or attributed to) Makron, Douris, and the Brygos Painter in a 10-fold cross validation with considerable accuracy. In a second step, annotations for the most prominent classes of the data set were extracted from each image to train a separate feature extractor in the same fashion. Finally, a deep set architecture (Zaheer et al. 2017) combines the full image and set of annotations, leading to promising results. We want to extend this approach by incorporating additional unlabelled examples in a semi-supervised fashion and joint training for automated localization of objects (Arif et al. 2019) instead of manual annotation to further improve the performance. Several methods of evaluating the CNN’s criteria are currently being used to further improve the accuracy, to develop the CNN’s architecture, and to reflect on the selection and structure of the data set. Furthermore, we are evaluating our approach through various methods for model interpretability and explainability such as Grad-cam (Selvaraju et al. 2017) for visualizing input gradients and classical feature selection approaches.

Our interdisciplinary approach builds on a deep intertwining of archaeology and computer science and leads to a mutual re-evaluation of traditional and innovative methods. The handling of small but highly complex data sets, and the comparison of the potential and limits of human experts vs. machine learning, contribute to current research and discussions in the field of digital humanities. In particular, the critical discussion of the criteria underlying painter attribution stimulates highly relevant socio-cultural research questions in the field of classical archaeology.