Cultural Heritage Imaging


Illumination of Material Culture: A Symposium on Computational Photography and Reflectance Transformation Imaging (RTI) at The Met, March 7-8, 2017 by chicaseyc

Our guest blogger, Emily B. Frank, is currently pursuing a Joint MS in Conservation of Historic and Artistic Works and MA in History of Art at New York University, Institute of Fine Arts. Thank you, Emily!

With the National Endowment for the Humanities (NEH) back on the chopping block in the most recent federal budget proposal, I feel particularly privileged to have taken part in the NEH-funded symposium, Illumination of Material Culture, earlier this month.

Co-hosted by The Metropolitan Museum of Art and Cultural Heritage Imaging (CHI), the symposium brought together conservators, curators, archaeologists, imaging specialists, cultural heritage and museum photographers, and the gamut of engineers to discuss and debate uses, innovations, and limitations of computational imaging tools. This interdisciplinary audience fostered an environment for collaboration and progress, and a few themes emerged.

The sold-out crowd at the symposium at The Met

The sold-out crowd at the symposium at The Met

(1) The emphasis among practitioners seems to have shifted from isolated techniques to integrating a range of data types.

E. Keats Webb, Digital Imaging Specialist at the Smithsonian’s Museum Conservation Institute, presented “Practical Applications for Integrating Spectral and 3D Imaging,” the focus of which was capturing and processing broadband 3D data. Holly Rushmeier, Professor of Computer Science at Yale University, gave a talk entitled “Analyzing and Sharing Heterogeneous Data for Cultural Heritage Sites and Objects,” which focused on CHER-Ob, an open source platform developed at Yale to enhance the analysis, integration, and sharing of textual, 2D, 3D, and scientific data. CHI’s Mark Mudge presented a technique for the integrated capture of RTI and photogrammetric data. The theme of integration propagated through the panelists’ presentations and the lightning talks, including but not limited to presentations by Kathryn Piquette, Senior Research Consultant and Imaging Specialist at University College London, on the integration of broadband multispectral and RTI data; Nathan Matsuda, PhD Candidate and National Science Foundation Graduate Fellow at Northwestern University, on work at NU-ACCESS with photometric stereo and photogrammetry; as well as a lightning talk by Chantal Stein, in collaboration with Sebastian Heath, Professor of Digital Humanities at the Institute for the Study of the Ancient World; and myself, about the integration of photogrammetry, RTI, and multiband data into a single, interactive file in Blender, a free, open source 3D graphics and animation software.

(2) There is an emerging emphasis on big data and the possibilities of machine learning.

Paul Messier, art conservator and head of the Lens Media Lab at the Institute for the Preservation of Cultural Heritage, Yale University

Paul Messier, art conservator and head of the Lens Media Lab at the Institute for the Preservation of Cultural Heritage, Yale University

The notion of machine learning and the possibilities it might unlock were addressed in multiple presentations, perhaps most notably in the “RTI: Beyond Relighting,” a panel discussion moderated by Paul Messier, Head of the Lens Media Lab, Institute for the Preservation of Cultural Heritage (IPCH), Yale University. Dale Kronkright presented work in progress at the Georgia O’Keeffe Museum in collaboration with NU-ACCESS that utilizes algorithms to track change to the surfaces of paintings, focusing on the dimensional change of metal soaps. Paul Messier briefly described the work being done at Yale to explore the possibilities for machine learning to work iteratively with connoisseurs to push data-driven research forward.

Mark Mudge, President of Cultural Heritage Imaging, leads a panel discussion

Mark Mudge, President of CHI, participates in a panel discussion

(3) The development of open tools for sharing and presenting computational data via the web and social media is catching up.

Graeme Earl, Director of Enterprise and Impact (Humanities) and Professor of Digital Humanities at the University of Southampton, UK, gave a keynote entitled “Open Scholarship, RTI-style: Museological and Archaeological Potential of Open Tools, Training, and Data,” which kicked off the discussion about open tools and where the future is heading. Szymon Rusinkiewicz, Professor of Computer Science at Princeton University, presented “Modeling the Past for Scholars and the Public,” a case study of a cross-listed Archaeology-Computer Science course given at Princeton in which students generated teaching tools and web content that provided curatorial narrative for visitors to the museum. CHI’s Carla Schroer presented new tools for collecting and managing metadata for computational photography. Roberto Scopigno, Research Director of the Visual Computing Lab, Consiglio Nazionale delle Richerche (CNR), Istituto di Scienza e Tecnologie dell’Informazione (ISTI), Pisa, Italy, delivered the keynote on the second day of the symposium about 3DHOP, a new web presentation and collaboration tools for computational data.

We had the privilege of hearing from Tom Malzbender, without whose work at HP Labs in the early 2000s this symposium would never have happened.

The keynotes at the symposium were streamed through The Met’s Facebook page. The other talks were recorded and will be available in three to four weeks. Enjoy!

Tom Malzbender, the inventor of RTI, at the podium

Tom Malzbender, the inventor of RTI, at the podium

Advertisements


Revealing Inscriptions, Reading Surface Texture, and Documenting Change with RTI at Sardis by chicaseyc
January 24, 2016, 3:47 pm
Filed under: Guest Blogger, On Location, Technology | Tags: , , , , , ,

Our guest blogger, Emily B. Frank, is a student conservator on the Sardis Archaeological Expedition. Currently she is pursuing a Joint MS in Conservation of Historic and Artistic Works and MA in History of Art at New York University, Institute of Fine Arts. Thank you, Emily!

Sardis, the capital city of the Lydian empire in the seventh and sixth centuries BC, is often best remembered for the invention of coinage. Remains of a monumental temple of Artemis, begun during the Hellenistic period and never finished, still stand tall today. In Roman times, the city was famous as one of the Seven Churches of Asia in the Book of Revelation. In the fourth and fifth centuries AD, Sardis boasted what is still the largest known synagogue in antiquity. Sardis flourished and continued to grow in the Late Roman period until its decline by the seventh century AD.

Archaeological excavations at Sardis began over a century ago and are currently led by Dr. Nicholas D. Cahill, professor of Art History at the University of Wisconsin, Madison. The excavated material is vastly diverse and the conservation efforts there equally so. Conservation this season, under Harral DeBauche, a third-year conservation student at New York University, Institute of Fine Arts, supported active excavation across over 1,000 years of antiquity and addressed a number of site preservation issues. RTI greatly benefited the conservators and archaeologists in a couple of significant ways.

Significant finds with extremely shallow incised designs/inscriptions and impressions were made legible with RTI. RTI was helpful in understanding a bronze triangle recovered from the corridor of a Late Roman house (Fig. 1).

TriangleSetupImage-Fig1

Figure 1: H-RTI setup used for small objects at Sardis.

The triangle is incised with three images of a female deity and a border of magical signs (Fig. 2). Its use is likely connected with religious ritual practice in Asia Minor between the third and sixth centuries AD.

Figure 2: RTI of the bronze triangle in cumulative unsharp masking viewing mode.

Figure 2: RTI of the bronze triangle in cumulative unsharp masking viewing mode.

The legibility of the inscriptions, aided by RTI, reinforced the connection between the triangle’s inscriptions and material and written sources. Two comparenda for the triangle, one from Pergamon and one from Apamea, were identified, and the magic symbols on the triangle were connected with rituals described in a the Greek Magical Papyri.  RTI also aided in decoding and documenting a lead curse tablet and in understanding the weave structure of bitumen basketry impressions.

Additionally, a multi-year biofilm removal project of the Artemis Temple at Sardis is currently underway, headed by Michael Morris and Hiroko Kariya, conservators in private practice. The removal of this biofilm is carried out by a six-day process. RTI was used experimentally to document the changes to the stone throughout the removal process (Fig. 3).

Figure 3: H-RTI setup used for documentation of the biocide removal project.

Figure 3: H-RTI setup used for documentation of the biocide removal project.

RTIs were taken before treatment, during treatment (day 3), and after treatment (day 7). Because the biocide continues to work for months after its application, a final RTI will be taken next summer. Initial comparison of the images showed no loss to the stone surface as a result of the biofilm removal process. All very exciting!

To find out more about the excavations at Sardis, see http://sardisexpedition.org/.



Imaging and Studying Stone Tools with RTI by chicaseyc
June 30, 2015, 11:01 pm
Filed under: Guest Blogger, On Location | Tags: , , ,

This is the second post by our guest blogger Dr. Leszek Pawlowicz, an Associate Practitioner in the Department of Anthropology, Northern Arizona University, Flagstaff, AZ, USA. He can be contacted at leszek.pawlowicz@nau.edu. Thank you again, Leszek! Even in the age of digital photography, archaeology still relies heavily on old-school hand-drawn illustrations for documenting artifacts, particularly for publication. It can be difficult, or even impossible, to get a single photograph that shows all of the artifact’s key details. In my area of interest, knapped stone tools, low relief in small-scale surface topography, high relief in large-scale topography, specular (reflective) materials, variations in material color and contrast across the surface, all conspire to make these artifacts difficult to photograph. A skilled illustrator is capable of creating a drawing that reveals far more detail than even a good standard photograph (Figure 1). But creating such a drawing requires experience, time, and money, often making it impractical.

Figure_1

Figure 1: Comparison of digital photograph and line drawing of knapped stone tool. Courtesy Lance Trask, http://lktrask-media.com/.

Reflectance Transformation Imaging (RTI) is a great way to document and image knapped stone tools. The ability to interactively modify the lighting angle, as well as mathematically manipulate the perceived interaction of light and surface, allows the viewer to see details difficult to photograph. It’s almost the equivalent of having the object in your hands, and turning it at various angles to the light to reveal details of its structure and manufacture. However, one drawback is that you can’t embed an RTI view in a paper or publication unless it’s in electronic format. Also, unlike a drawing that can show all the critical details at once, you may have to move the virtual lighting angle in many different directions to reveal all details. Using data from the custom RTI systems I described in a previous post, I’ve been working on ways to create static images of knapped stone tools with detail comparable to that in a line drawing.

Figure_2-med

Figure 2: Modern obsidian knapped point. A – Original digital photograph; B – Static RTI view; C- RTI Specular mode; D – RTI Static Multilight mode; E – RTI Normals mode; F – Enhanced RTI Normals mode. Click on image for enlarged view.

Figure 2 shows a series of images of a modern knapped projectile point fashioned from specular black obsidian, a particularly difficult material to photograph because of its shininess and lack of contrast. 2-A is an original digital photograph, with the lighting coming from the standard upper left direction; while a fair amount of detail is visible, shiny highlights obscure some details, and the overall convex shape of the point interferes with lighting on the right side of the point. 2-B shows an RTI view with lighting from the upper left; while glossy highlights have been eliminated, and more detail is visible on the right, other details are somewhat more subdued in this static view. Figure 2-C uses the RTI specular viewing mode, which imparts an artificially shiny character to the surface. This is actually a superior result for this mode on knapped stone tools – most of the time, it doesn’t look nearly this good (as you’ll see shortly). But this image suffers from lack of detail in some areas, probably because of the artifact’s overall convex surface shape. Figure 2-D uses the RTI Static Multilight mode, where the RTI data is analyzed to determine the optimal blend of multiple light angles to reveal details. Once again, this is a better result than I usually get for knapped stone tools, but some details are still hard to see because of lack of contrast in many areas. Overall, I have found that the recently added RTI Normals viewing mode leads to the best results. This mode color-codes the surface based on the perpendicular direction at every point in the image. Figure 2-E was generated using the Normals mode and a second-order HSH RTI file (standard Polynomial Texture Mapping or PTM) files produce markedly poorer results). Even in the raw colored state, the amount of detail visible across the entire surface is vastly superior to the other views. Using standard image enhancement techniques, one can further process this image to generate an extremely detailed view of the artifact’s surface details (Figure 2-F). While this modern artifact was made of a difficult material, the freshness of manufacture, lack of wear, and non-exposure to the elements make the surface features quite sharp and easy to see. What about an actual prehistoric point with a real history?

Figure 3: Molina Spring Clovis point. A – Original digital photograph; B – Static RTI view; C- RTI Specular mode; D – RTI Static Multilight mode; E – RTI Normals mode; F – Enhanced RTI Normals mode; G – Slope-shaded mode. Click on image for enlarged view.

Figure 3: Molina Spring Clovis point. A – Original digital photograph; B – Static RTI view; C- RTI Specular mode; D – RTI Static Multilight mode; E – RTI Normals mode; F – Enhanced RTI Normals mode; G – Slope-shaded mode. Click on image for enlarged view.

Figure 3 shows a series of similar photographs for the Molina Spring Clovis point, collected about 10 years ago in the Apache-Sitgreaves National Forest. Clovis points are approximately 13,500 years old; this one in particular has seen a lot of wear on the flake scars, making them difficult to make out. What’s more, the point’s material (chert) is slightly glossy and very light in color, minimizing contrast in surface details. Figures 3-A through 3-F show the same image processing sequence as Figures 2-A through 2-F; note in particular that the Specular and Static Multilight modes (3-C and 3-D) do not produce especially useful images for this point. The raw Normals image (3-E) brings up details difficult to impossible to spot in the previous images, and enhancing the Normals image (3-F) makes those details even easier to see. For Figure 3-G, a Matlab script was used generate a modified view based on slope; this brings out certain details not immediately visible in the original normals image. In my opinion, images like 3-F and 3-G are superior to standard digital photographs of knapped stone tools and could well be used instead of line drawings for publication purposes. Even in cases where line drawings are preferred, these images could provide a useful background basis for tracing artifact details, instead of drawing them freehand. Beyond static images, the normals data can be used to generate a full 3D representation of the artifact’s surface. Some examples of this are visible at my website, along with downloadable 3D files and RTI data files for several projectile points. The 3D surfaces aren’t yet fully accurate, as they can be affected by inaccuracies in the normals calculation, and error accumulations in the surface fitting. However, these results are already significantly improved from my initial efforts, and I will be working on improving them further. Even in the current form, I believe that it should be possible to extract usable information than can be used to quantitatively characterize knapped stone tools.



Creating a Portable Dome-RTI system for Imaging Lithics by chicaseyc

Our guest blogger is Dr. Leszek Pawlowicz, an Associate Practitioner in the Department of Anthropology, Northern Arizona University, Flagstaff, AZ, USA. He can be contacted at leszek.pawlowicz@nau.edu. A longer version of this post can be seen at http://rtimage.us/?page_id=27. Thank you, Leszek!

When I learned about Reflectance Transformation Imaging (RTI) back in 2009, one of my first thoughts was that it could be a useful tool for imaging and analyzing lithic archaeological artifacts, flaked stone tools in particular. Not an original thought even back then, and over the next four years I’ve seen the occasional RTI lithic image pop up on the web, demonstrating how useful RTI could be in this application. Early in 2013, I started experimenting with RTI on some modern replica projectile points using Highlight-RTI method. Though I got usable results with these experiments, I decided that Dome-RTI was a more appropriate method because of the reduced data acquisition and processing times.

So began a two-year process of building my first Dome-RTI system and refining it. After multiple iterations of the lighting system, controller, and camera/dome stand, I wound up with an 18″-diameter acrylic dome that produces excellent results and is useful for RTI on larger artifacts. However, it’s grossly over-sized for most of the artifacts I’m interested in documenting. Most flaked stone lithic artifacts in the American Southwest are less than 3 inches in length, and an 18″ dome is easily capable of imaging artifacts of at least 4.5″ in maximum dimension (I’ve gotten useful results on artifacts up to 6″ in length). What’s more, these artifacts are housed in scattered locations (museums, government facilities, universities, etc.), and the large size of the dome and stand make transportation and setup of this big system cumbersome. So, applying lessons learned from the first system, I built a second system with an emphasis on portability and speed (Figure 1):

portable RTI dome

Figure 1: Portable RTI dome

  • Dome diameter is 12″, and sits on a stand that is 13.5″ square; total weight of the dome + stand + camera is less than 4 kg. The small size lets it fit into a Pelican case for easy transport.
  • The controller box automatically lights 48 3W LEDs in sequence for the light sources; maximum current is 1 amp, and can be set as low as 150 milliamps. The camera shutter is triggered automatically in sync with the LEDs using either a wired remote cable, an IR remote signal, or a Bluetooth HID transmitter; a manual shutter mode is also available.
  • Data acquisition time is about 3 minutes with a Canon S110 camera (12 MP, native 12-bit RAW), about one minute with Canon/Nikon DSLRs. A custom GUI front-end for the PTM and HSH fitters reduces data processing time to 1-3 minutes after the photographs are transferred to a computer.
  • Dome is mounted on a hinged stand, which allows artifacts to be swapped in/out in about 10 seconds.
  • Entire system is powered by 9-12V DC, either from a wall transformer or appropriate battery power supply.

The system can fit securely in a standard camping backpack with room to spare, with a total weight of less than 5 kg. The option of battery power makes this a truly portable, field-ready RTI system (Figure 2).

Portable RTI Dome “in the field”, north of San Francisco Peaks, Flagstaff, Arizona

Figure 2: Portable RTI Dome “in the field,” north of San Francisco Peaks, Flagstaff, Arizona

When recording archaeological sites out in the field, it is often not possible to collect lithic artifacts to bring back to the lab for proper documentation. You either have to photograph them in the field (usually with less-than-satisfactory resolution of artifact details), or hand-draw the flake scars (a slow and tedious process, and often highly inaccurate). This portable RTI system makes it possible to thoroughly document lithic artifacts on-site.

This system has a few more tricks up its sleeve. Full analysis of a lithic artifact may require microscopic analysis of edgewear to determine how it was used.

Dome in microphotography mode with USB microscope

Figure 3: Dome in microphotography mode with USB microscope

A simple reconfiguration of the system (Figure 3) allows high-magnification RTI imaging of lithic artifacts, using either the USB microscope (as pictured), or a DSLR equipped with a macro lens that has a working distance of 6″ or more (roughly 90-100mm focal length). A micrometer stage allows for accurate positioning of the artifact under the microscope.

You can also reconfigure the stand to mount the dome vertically for imaging larger artifacts. While I plan to use it in this mode to image the surface of Southwestern pottery, Figure 4 shows the system in vertical mode being used to image an oil painting.

Dome in vertical mode, imaging oil painting

Figure 4: Dome in vertical mode, imaging an oil painting

The normals may be off a bit because of the increased spacing between dome and painting, but you can still get useful results, like the specular mode image shown in Figure 5.

RTI specular image of painting surface shot in vertical mode

Figure 5: RTI specular image of painting surface shot in vertical mode

Total parts cost of this portable RTI dome, including the Canon camera, was well under $800. Scaling the dome up to a higher size would increase the expenditure by only the extra cost of the dome plus additional LEDs if desired (e.g. 64 instead of 48). For example, a one-meter dome with 64 LEDs would add approximately $400 to the total cost. In the near future, I hope to post information/instructions online that would allow anyone to build a system of their own. If I can build a system without instructions, I’m sure many others could easily build such a system with instructions.

In an upcoming post, I’ll present some of my lithics RTI imaging results from both of my Dome-RTI systems.



Behind the Scenes: Museum Photography at the Oriental Institute by marlinlum

Recently Anna R. Ressman, Head of Photography at the Oriental Institute Museum, University of Chicago, shared a compelling article with me, and now I’m sharing it with you.

Here is a link to the Oriental Institute newsletter (PDF), which contains the article entitled, “Behind the Scenes: Museum Photography at the Oriental Institute.

Anna describes the process in which five very different artifacts are documented, each with a unique challenge. And yes, you guessed it, one of those artifacts was documented using the RTI highlight method.

Documentation of the Egyptian stele “was photographed with a method of computational photography called Reflectance Transformation Imaging (RTI).”

Using RTI to photograph OIM E14655 with photo assistants K. Bryce Lowry and Austin M. Kramer

Using RTI to photograph OIM E14655 with photo assistants K. Bryce Lowry and Austin M. Kramer

OIM E14655, Egyptian Stele, Limestone, New Kingdom, Medinet Habu, Egypt. 36x26cm

OIM E14655, Egyptian Stele, Limestone, New Kingdom, Medinet Habu, Egypt. 36x26cm

Specular enhancement using RTI of Egyptian Stele OIM E14655

Specular enhancement using RTI of Egyptian Stele OIM E14655

Anna concludes the section on RTI with these insights: “RTI files can be created in such a manner that pixel data is analyzed to show specular information rather than color data, which can reveal more information about the surface of the object than color data alone (figs. 3–4). As you can see, the inscriptions on the stele are much clearer in the specular-enhancement PTM image (fig. 3), even though the studio photograph (fig. 4) was taken using a macro lens under controlled studio lighting. The former may not be as aesthetically pleasing as the latter, but it reveals much more information than would normally be seen — and that is just a single image out of a series of forty-five.”

Be sure to download the complete article and check out the rest of the newsletter as well.

Anna R. Ressman is Head of Photography at the Oriental Institute Museum, University of Chicago, USA. Anna is also a freelance photographer and a fine artist.

[Photos by Anna R. Ressman/Courtesy Oriental Institute Museum, University of Chicago]



Greek red-figure vases, two surface examination methods and fabricated mock-ups by marlinlum

A linierhaar made of human hair is used to produce looped laid lines similar to that seen in Fig. 10 (left) (photo by Kari Kipper). A 3D elevation map of one such fabricated looped line displays topographical features distinctly similar to the ancient looped line (right). The threshold setting in the elevation map was adjusted to remove the majority of measurements associated with the cardboard substrate. The dimensions of the elevation map are 1.73 × 2.9 mm. The elevation scale bar is in μm.

Paula Artal-Isbrand and Philip Klausmeyer recently published an article in the Studies in Conservation journal.

Entitled “Evaluation of the relief line and the contour line on Greek red-figure vases using reflectance transformation imaging and three-dimensional laser scanning confocal microscopy,” the article examines “…the relief and contour lines on a group of ancient Greek red-figure vases and vase fragments.”

Paula and Philip, both of the Worcester Art Museum in Worcester, Massachusetts, USA, describe how they deployed “… two surface examination methods – reflectance transformation imaging and three-dimensional laser scanning confocal microscopy” to “… characterize the lines and answer questions regarding tools, techniques, and production sequence used by Greek vase painters.”

Their work is interesting and empirical, with numerous examples that yield detailed observations about the tools and techniques used to create the decorative features on vases and vase fragments, with a particular emphasis on relief and contour lines.

Download the entire PDF:

Evaluation of the relief line and the contour line on Greek red-figure vases using reflectance transformation imaging and three-dimensional laser scanning confocal microscopy

 



CHI Welcomes Tom Malzbender to the Board of Directors! by cdschroer
March 21, 2014, 4:02 pm
Filed under: Commentary, News, Technology | Tags: , ,
Tom Malzbender imaging a piece from the Antikythera Mechanism

Tom Malzbender imaging a piece from the Antikythera Mechanism

We are absolutely thrilled to announce that Tom Malzbender has joined the Cultural Heritage Imaging Board of Directors!

Tom is a long-time friend of CHI and has been an adviser and collaborator on many projects. Tom is best known as the co-inventor (with Dan Gelb) of Polynomial Texture Mapping (PTM) in 2001 while he was at HP Labs. PTM is the first form of Reflectance Transformation Imaging (RTI).

Tom has had a long research career in  the fields of computer graphics and computer vision as demonstrated by his many published papers.  He has frequently applied this work to cultural heritage material, most notably as part of the team that deciphered the Antikythera Mechanism. This groundbreaking work was featured in the NOVA documentary  “Ancient Computer”

Welcome Tom!