Ingrid Daubechies, Pier Luigi Dragotti, Nathan Daly, Catherine Higgitt and Miguel Rodrigues
Duke University, National Gallery, University College London
The cultural heritage sector is experiencing a digital revolution driven by the growing adoption of non-invasive, non-destructive spectroscopic imaging approaches generating multi-dimensional data from entire artworks. Such approaches include ‘macro X-ray fluorescence’ (MA-XRF) scanning or hyper-spectral imaging (HSI) in the visible and infrared ranges and are highly complementary to more traditional broad band digital imaging techniques such as X-ray radiography (XRR) or infrared reflectography (IRR).
This data –spanning both the spatial and spectral domains– holds information about both materials at the surface of an artwork but also about sub-surface layers or features of interest otherwise invisible to the naked eye. The ability to interrogate the wealth of data yielded by these techniques can potentially provide insights into an artist’s materials, techniques and creative process; reveal the changing condition of an artwork over time and its restoration history; help inform strategies for the conservation and preservation of artworks; and, importantly, offer means by which to present artwork to the public in new ways.
However, to do this successfully also calls for new sophisticated signal and image processing tools capable of addressing various challenges associated with the analysis, interrogation, and processing of such massive multi-dimensional datasets. These challenges derive from the fact that paintings are very complex objects where
- Materials are often present in intimate mixtures applied over multi-layered systems so signals deriving from spectroscopic imaging techniques are highly nonlinearly mixed.
- Materials also age/degrade over time so signals collected from spectroscopic imaging techniques cannot be often compared to signals present in reference libraries.
- A ‘ground-truth’ is often unavailable or limited because each painting is unique, with original materials often unknown, and materials’ aging process also unknown.
In addition, different spectroscopic imaging techniques reveal different details about artwork, so there is also a need to develop new signal and image processing tools combining different datasets in order to understand artwork.
This tutorial –which is offered by experts in applied mathematics, signal & image processing, machine learning, and heritage science– (a) reviews the state-of-the-art in signal and image processing for art investigation (b) reviews signal and image processing challenges arising in the examination of datasets acquired on artwork and (c) overviews emerging directions in signal processing for art investigation.