Filed under: Conferences, Grants, Guest Blogger, Technology | Tags: 3D data, CHI, conservation, Digital Imaging, Digital Preservation, Open Source, RTI, symposium
Our guest blogger, Emily B. Frank, is currently pursuing a Joint MS in Conservation of Historic and Artistic Works and MA in History of Art at New York University, Institute of Fine Arts. Thank you, Emily!
With the National Endowment for the Humanities (NEH) back on the chopping block in the most recent federal budget proposal, I feel particularly privileged to have taken part in the NEH-funded symposium, Illumination of Material Culture, earlier this month.
Co-hosted by The Metropolitan Museum of Art and Cultural Heritage Imaging (CHI), the symposium brought together conservators, curators, archaeologists, imaging specialists, cultural heritage and museum photographers, and the gamut of engineers to discuss and debate uses, innovations, and limitations of computational imaging tools. This interdisciplinary audience fostered an environment for collaboration and progress, and a few themes emerged.
(1) The emphasis among practitioners seems to have shifted from isolated techniques to integrating a range of data types.
E. Keats Webb, Digital Imaging Specialist at the Smithsonian’s Museum Conservation Institute, presented “Practical Applications for Integrating Spectral and 3D Imaging,” the focus of which was capturing and processing broadband 3D data. Holly Rushmeier, Professor of Computer Science at Yale University, gave a talk entitled “Analyzing and Sharing Heterogeneous Data for Cultural Heritage Sites and Objects,” which focused on CHER-Ob, an open source platform developed at Yale to enhance the analysis, integration, and sharing of textual, 2D, 3D, and scientific data. CHI’s Mark Mudge presented a technique for the integrated capture of RTI and photogrammetric data. The theme of integration propagated through the panelists’ presentations and the lightning talks, including but not limited to presentations by Kathryn Piquette, Senior Research Consultant and Imaging Specialist at University College London, on the integration of broadband multispectral and RTI data; Nathan Matsuda, PhD Candidate and National Science Foundation Graduate Fellow at Northwestern University, on work at NU-ACCESS with photometric stereo and photogrammetry; as well as a lightning talk by Chantal Stein, in collaboration with Sebastian Heath, Professor of Digital Humanities at the Institute for the Study of the Ancient World; and myself, about the integration of photogrammetry, RTI, and multiband data into a single, interactive file in Blender, a free, open source 3D graphics and animation software.
(2) There is an emerging emphasis on big data and the possibilities of machine learning.
The notion of machine learning and the possibilities it might unlock were addressed in multiple presentations, perhaps most notably in the “RTI: Beyond Relighting,” a panel discussion moderated by Paul Messier, Head of the Lens Media Lab, Institute for the Preservation of Cultural Heritage (IPCH), Yale University. Dale Kronkright presented work in progress at the Georgia O’Keeffe Museum in collaboration with NU-ACCESS that utilizes algorithms to track change to the surfaces of paintings, focusing on the dimensional change of metal soaps. Paul Messier briefly described the work being done at Yale to explore the possibilities for machine learning to work iteratively with connoisseurs to push data-driven research forward.
(3) The development of open tools for sharing and presenting computational data via the web and social media is catching up.
Graeme Earl, Director of Enterprise and Impact (Humanities) and Professor of Digital Humanities at the University of Southampton, UK, gave a keynote entitled “Open Scholarship, RTI-style: Museological and Archaeological Potential of Open Tools, Training, and Data,” which kicked off the discussion about open tools and where the future is heading. Szymon Rusinkiewicz, Professor of Computer Science at Princeton University, presented “Modeling the Past for Scholars and the Public,” a case study of a cross-listed Archaeology-Computer Science course given at Princeton in which students generated teaching tools and web content that provided curatorial narrative for visitors to the museum. CHI’s Carla Schroer presented new tools for collecting and managing metadata for computational photography. Roberto Scopigno, Research Director of the Visual Computing Lab, Consiglio Nazionale delle Richerche (CNR), Istituto di Scienza e Tecnologie dell’Informazione (ISTI), Pisa, Italy, delivered the keynote on the second day of the symposium about 3DHOP, a new web presentation and collaboration tools for computational data.
We had the privilege of hearing from Tom Malzbender, without whose work at HP Labs in the early 2000s this symposium would never have happened.
The keynotes at the symposium were streamed through The Met’s Facebook page. The other talks were recorded and will be available in three to four weeks. Enjoy!
I have had the good fortune to attend a few recent events that allowed me to see some really useful work other folks are doing in our field. I thought it worth a blog post to mention a few with some links. I’ll note that I have seen even more cool stuff, but if there wasn’t a paper or a page I could link to, I decided not to include it here.
First, Mark Mudge and I were at Computer Applications and Quantitative Methods in Archaeology (CAA) in late April in Paris. Yes, I know, it is a hard-knock life having to present your work in Paris in April. I always love CAA because it is a great conference for sharing information and real-life experiences. People are super helpful, and everyone wants to see everyone else succeed. This isn’t the norm for a lot of conferences.
I want to point out 2 projects from there. The first is Eleni Kotoula’s RTI on Papyrus case study. New here is Eleni’s use of multispectral imaging and also her experiment with transmitted RTI. Eleni was interested in information for conservation about the state of this papyrus, which is mounted on cloth and under glass. The second project at CAA isn’t an RTI project, but it’s very cool and worth mentioning anyway. It is the work of Adam Rabinowitz et al. on PeriodO. As folks who follow our efforts know, we at CHI are big fans of metadata, process-history tracking, and related topics. Figuring out how to talk about periods in archaeology and art is really hard, and Adam and team have an idea for an interesting approach. This project recently won National Endowment for the Humanities support, so it will be able to go forward. You can and should contribute to this effort yourself!
Next up is the American Institute for Artistic and Historic Works (AIC) conference, which took place at the end of May, right here in San Francisco. It was great for us to have so many folks we have worked with here in SF. One of the things I noticed at this conference was that Reflectance Transformation Imaging (RTI) showed up in a number of talks, though that wasn’t the focus of the talk. RTI is another tool to look at objects people are studying, and RTIs are being shown right alongside IR images and X-rays and the like. YAY! My takeaway is that RTI is an accepted part of the practice for art conservators. We are super happy to see that.
Finally, I was able to attend a 3-day workshop on RTI current practice and future directions, hosted by the Cyprus Institute in Nicosia, Cyprus in early June. This was “All RTI, All the Time!” and it was very productive to be part of a dedicated meeting like this. There were many noteworthy projects presented there, and I’ll point out a few. Up first, the work of Todd Hanneken and others integrating spectral imaging and RTI. While many of us have done spectral imaging and RTI going back several years, what is new here is the use of a MegaVision monochrome camera system, and taking 11 spectral bands. RTI was shot in the visible and used for luminance data to generate the RTIs, and the color data collected via the spectral bands was then applied to the images shot in visible light. The team also tested shooting full RTIs in each spectral band for comparison. In addition, work from the Visual Computing Lab in Pisa for a WebGL-based RTI viewer, released in January of this year, was discussed. At the moment, this work is limited to streaming large RTI files and then allowing the user to pan, zoom, and relight, but some possible future directions were outlined. It is open source, so you can try it out yourself. There is also a project at The University of Southampton to develop a web-based RTI viewer, and we received an update on that project, but I don’t have a link where I can send you just yet.
As part of the discussion on where RTI is going, there was a lot of interest in quantitative uses of RGB and normals data. There are a variety of people working in this area, though they didn’t present directly at any of these meetings. Several of these folks are working with us at CHI, and we are very excited by this direction. First up, the work of Dale Kronkright, Greg Bearman, and several others to look at tracking changes through normals, and also to quantify normal calibration. You can find both papers here. Additionally, there is great work going on at Simon Fraser University under Professor Mark Drew to improve the accuracy of surface normals calculated from RTI data sets. They are also working on improving the appearance of RTI data in the viewing environment. There are a number of papers on this topic, most recently the masters thesis of Mingjing Zhang.
It is an exciting time to be working with RTI data! There is much more work going on than I could include here, and more that hasn’t been published yet. I want to close with a plug for the CHIForums where many topics like this are discussed, and there is a Projects Forum where folks can put up links to their work, as well as get comments and feedback.
During the month of May we had the pleasure of doing more imaging work with Rock Art . This has included shooting some Reflectance Transformation Imaging (RTI) and some photogrammetry sequences at a couple of different sites. More importantly, we have had a chance to present some of this work to folks researching and recording rock art. We presented a workshop at theIFRAO conference in Albuquerque a couple of weeks ago. This was our first time attending the international conference. We were able to go to papers on all aspects of rock art research from all over the world. One of the things I love about rock art is that there are some things we just can’t know particularly about the older material where we don’t have living descendants from the culture to help us understand it. I find joy in that mystery.
One thing that is really clear is that rock art sites all over the planet are at risk and a lot of rock art is being lost every year. This is due to a wide range of factors from vandalism, to development, to earthquakes, to flooding and fires, to things as simple as natural rock falls. Part of our mission at CHI is to get tools to document these sites into the hands of folks who care about them. It is increasingly clear to me that teaching people how to capture the photographic image sequences that will allow the generation of full 3 D models, could really help us have records for future generations, and could provide a baseline of the current state of sites that would help monitor the changing conditions. The great thing about this approach is that it just takes a digital camera (and a monopod helps) and a bit of training about how to take the images in order to ensure complete coverage and high quality results. There are a number of different commercial packages available, and also some open source efforts, and the great thing is that how you capture the images is the same no matter where you want to process the images to get high quality 3D. We are now working on developing photogrammetry training for folks working in archaeology, historic sites, rock art sites, and related fields. Even if the data isn’t all processed in the short term, archiving the photographs that are properly collected will mean that anyone can create the 3D models in the future. To be clear, I don’t think anything takes the place of being in an actual site, and that it is critically important to protect and preserve these sites. But, given the fragility of these places, and also the inaccessibility of many of them, we should be gathering as much high quality data as we can as inexpensively as we can for ourselves and for future generations.
June 8, 2011
At the recent annual meeting of the American Institute for Conservation (AIC) in Philadelphia, I was happy to see both past and future participants in the IMLS-sponsored CHI RTI training session program. I had to break the news to some interested parties that the sessions at the Worcester Art Museum and at the San Francisco Museum of Modern Art are already filled (with waiting lists) – but a few openings remain (apply here) in the sessions scheduled for the Smithsonian’s Museum Conservation Institute and the Indianapolis Museum of Art. Since AIC, several spots have already been taken.
Of course, conservators always want more visual information, and have been quick to understand the benefits of RTI for their work. Once conservators attend a CHI training session, RTI adoption spreads throughout the conservation community. At AIC, I heard several tales of conservators reaching out to their curatorial colleagues, presenting the extremely detailed technical art historical information to be gained from RTI. Curators were impressed! The ability to acquire this kind of detail is one of the hallmarks of RTI – in fact, in CHI’s Kress video Debbie Evans (paper conservator, Fine Arts Museums of San Francisco) mentions that RTI can allow the conservator to provide this important, detailed information to the curator. We look forward to more wonderful conservator/curator interactions!
By Elizabeth Peña
Just before the memorial day weekend Mark and I had the pleasure of attending the Society for the Preservation of Natural History Collections(SPNHC) conference. A national conference (and a few international folks too) held here in San Francisco this year. We gave an oral presentation and a demo camp presentation. Both were well received and afforded us the opportunity to speak to folks from various institutions. We have done a lot of work with fine arts institutions, but only a few proof of concept projects with natural history collections. It was a real treat to have a chance to go to some talks and panel discussions and also a collections tour at CalAcademy. I have always thought natural history museums were interesting and important and cool, but this conference really brought home to me just how important these collections really are as a scientific record of our planet. Collections large and small are critical to helping us answer questions about the natural world both now and in the future. A great example I saw on the collections tour were finches collected in 1905 – 1906 from the Galapagos, where the preserved skins allowed current researchers to do DNA analysis to answer new questions. Since DNA analysis didn’t exist when the specimens were collected, this clearly wasn’t something they were trying to enable. And yet, if we do a good job at collecting and preserving things (objects, data, specimens, etc.) then they just get richer and more valuable over time. That is one of our goals at CHI, to make data increase in value over time by collecting high quality imaging data and good records of how it was collected and what was done to it.
It was a great experience to be at SPNHC, and we hope to attend next year when it is hosted by the Peabody Museum at Yale.
A couple of weeks ago Carla Schroer conducted a workshop for a selected group of members of the Western Association for Art Conservation (WAAC) and members of cultural institutions in the Portland, OR area. The workshop was conducted at the beautifully restored and technologically advanced White Stag building of the University of Oregon, Portland. The title of the workshop was Digital Imaging Techniques for Conservation & Education. The three hour workshop was divided in two main sessions. In the first session, Carla lecture, via presentations, on the development and implementation of Reflectance Transformation Imaging (RTI) in the conservation field and cultural heritage institutions.
Carla brought a small fossil with her to be photographed. She explained the minimum equipment requirements and recommendations for successful setups for different size and types of objects. Once all the mechanics where explained Carla, a volunteer from the group, and my self proceeded to capture the object with 24 images each with a different lighting position. She followed by explaining the postproduction steps to build the final RTI image as well as the use of available viewers for these images. She also demonstrated some of the computational enhancements that could be applied to the images and the amazing amount of information that can be obtained in this way. The day ended with a great Q&A session that really brought home many of the concepts that Carla had touch upon during the day finalizing a really informative workshop.
On a personal note I am very happy to have had the opportunity to assist Carla with the practical aspect of the workshop, but even more happy that Carla came to the meeting and that she was able to mingle and meet many of the members of this wonderful WAAC organization. Wishing a long lasting partnership and collaboration among the two institutions I remain thankful.
Sr. Conservation Photographer
Conservation Center, LACMA
The Virutal Reality, Archaeology and Cultural Heritage (VAST) conference was great for us, and we are still enjoying being in Paris. This is a small focused conference with usually about 100 people in attendance. We put together a full day tutorial at the start of the conference with some of our favorite collaborators, and it was great to get this group together in person. We covered a lot of ground in a day with a nice mix of practical and theoretical material.
The rest of the conference was really good. I always enjoy the chance to catch up with various colleagues and friends, meet new folks working in this field, and see what other folks are up to. VAST is usually more on the computer science side of things, but there were a number of museum folks as well as archaeologists in attendance. There are always lively and sometimes controversial discussions, and this year lived up to that standard. Of great interest to me were the papers and discussions around how digital representations can track and reflect their true digital provenance from acquisition through to the finished process.
I particularly appreciated a discussion with Holly Rushmeier from Yale University about what we mean by accuracy, high resolution, and quality in general? In particular what is even measurable, and how should we be measuring and recording it? Holly agreed that this is an area that needs some attention, and it will take the work of multiple institutions working together to come to any guidelines.
Having a chance to talk to Martin Doerr about mapping the digital provenance data to the CIDOC CRM always requires having your thinking cap on, but it’s so worth it. There were many other great people in attendance, interesting papers, and good food, wine and conversation. I look forward to next year when VAST is combined with VSMM (International Society on Virtual Systems and Multi-Media) in Alexandria, Egypt.