Filed under: Conferences, Grants, Guest Blogger, Technology | Tags: 3D data, CHI, conservation, Digital Imaging, Digital Preservation, Open Source, RTI, symposium
Our guest blogger, Emily B. Frank, is currently pursuing a Joint MS in Conservation of Historic and Artistic Works and MA in History of Art at New York University, Institute of Fine Arts. Thank you, Emily!
With the National Endowment for the Humanities (NEH) back on the chopping block in the most recent federal budget proposal, I feel particularly privileged to have taken part in the NEH-funded symposium, Illumination of Material Culture, earlier this month.
Co-hosted by The Metropolitan Museum of Art and Cultural Heritage Imaging (CHI), the symposium brought together conservators, curators, archaeologists, imaging specialists, cultural heritage and museum photographers, and the gamut of engineers to discuss and debate uses, innovations, and limitations of computational imaging tools. This interdisciplinary audience fostered an environment for collaboration and progress, and a few themes emerged.
The sold-out crowd at the symposium at The Met
(1) The emphasis among practitioners seems to have shifted from isolated techniques to integrating a range of data types.
E. Keats Webb, Digital Imaging Specialist at the Smithsonian’s Museum Conservation Institute, presented “Practical Applications for Integrating Spectral and 3D Imaging,” the focus of which was capturing and processing broadband 3D data. Holly Rushmeier, Professor of Computer Science at Yale University, gave a talk entitled “Analyzing and Sharing Heterogeneous Data for Cultural Heritage Sites and Objects,” which focused on CHER-Ob, an open source platform developed at Yale to enhance the analysis, integration, and sharing of textual, 2D, 3D, and scientific data. CHI’s Mark Mudge presented a technique for the integrated capture of RTI and photogrammetric data. The theme of integration propagated through the panelists’ presentations and the lightning talks, including but not limited to presentations by Kathryn Piquette, Senior Research Consultant and Imaging Specialist at University College London, on the integration of broadband multispectral and RTI data; Nathan Matsuda, PhD Candidate and National Science Foundation Graduate Fellow at Northwestern University, on work at NU-ACCESS with photometric stereo and photogrammetry; as well as a lightning talk by Chantal Stein, in collaboration with Sebastian Heath, Professor of Digital Humanities at the Institute for the Study of the Ancient World; and myself, about the integration of photogrammetry, RTI, and multiband data into a single, interactive file in Blender, a free, open source 3D graphics and animation software.
(2) There is an emerging emphasis on big data and the possibilities of machine learning.
Paul Messier, art conservator and head of the Lens Media Lab at the Institute for the Preservation of Cultural Heritage, Yale University
The notion of machine learning and the possibilities it might unlock were addressed in multiple presentations, perhaps most notably in the “RTI: Beyond Relighting,” a panel discussion moderated by Paul Messier, Head of the Lens Media Lab, Institute for the Preservation of Cultural Heritage (IPCH), Yale University. Dale Kronkright presented work in progress at the Georgia O’Keeffe Museum in collaboration with NU-ACCESS that utilizes algorithms to track change to the surfaces of paintings, focusing on the dimensional change of metal soaps. Paul Messier briefly described the work being done at Yale to explore the possibilities for machine learning to work iteratively with connoisseurs to push data-driven research forward.
Mark Mudge, President of CHI, participates in a panel discussion
(3) The development of open tools for sharing and presenting computational data via the web and social media is catching up.
Graeme Earl, Director of Enterprise and Impact (Humanities) and Professor of Digital Humanities at the University of Southampton, UK, gave a keynote entitled “Open Scholarship, RTI-style: Museological and Archaeological Potential of Open Tools, Training, and Data,” which kicked off the discussion about open tools and where the future is heading. Szymon Rusinkiewicz, Professor of Computer Science at Princeton University, presented “Modeling the Past for Scholars and the Public,” a case study of a cross-listed Archaeology-Computer Science course given at Princeton in which students generated teaching tools and web content that provided curatorial narrative for visitors to the museum. CHI’s Carla Schroer presented new tools for collecting and managing metadata for computational photography. Roberto Scopigno, Research Director of the Visual Computing Lab, Consiglio Nazionale delle Richerche (CNR), Istituto di Scienza e Tecnologie dell’Informazione (ISTI), Pisa, Italy, delivered the keynote on the second day of the symposium about 3DHOP, a new web presentation and collaboration tools for computational data.
We had the privilege of hearing from Tom Malzbender, without whose work at HP Labs in the early 2000s this symposium would never have happened.
The keynotes at the symposium were streamed through The Met’s Facebook page. The other talks were recorded and will be available in three to four weeks. Enjoy!
Tom Malzbender, the inventor of RTI, at the podium
I have had the good fortune to attend a few recent events that allowed me to see some really useful work other folks are doing in our field. I thought it worth a blog post to mention a few with some links. I’ll note that I have seen even more cool stuff, but if there wasn’t a paper or a page I could link to, I decided not to include it here.
First, Mark Mudge and I were at Computer Applications and Quantitative Methods in Archaeology (CAA) in late April in Paris. Yes, I know, it is a hard-knock life having to present your work in Paris in April. I always love CAA because it is a great conference for sharing information and real-life experiences. People are super helpful, and everyone wants to see everyone else succeed. This isn’t the norm for a lot of conferences.
Musee de Cluny, Paris
I want to point out 2 projects from there. The first is Eleni Kotoula’s RTI on Papyrus case study. New here is Eleni’s use of multispectral imaging and also her experiment with transmitted RTI. Eleni was interested in information for conservation about the state of this papyrus, which is mounted on cloth and under glass. The second project at CAA isn’t an RTI project, but it’s very cool and worth mentioning anyway. It is the work of Adam Rabinowitz et al. on PeriodO. As folks who follow our efforts know, we at CHI are big fans of metadata, process-history tracking, and related topics. Figuring out how to talk about periods in archaeology and art is really hard, and Adam and team have an idea for an interesting approach. This project recently won National Endowment for the Humanities support, so it will be able to go forward. You can and should contribute to this effort yourself!
Next up is the American Institute for Artistic and Historic Works (AIC) conference, which took place at the end of May, right here in San Francisco. It was great for us to have so many folks we have worked with here in SF. One of the things I noticed at this conference was that Reflectance Transformation Imaging (RTI) showed up in a number of talks, though that wasn’t the focus of the talk. RTI is another tool to look at objects people are studying, and RTIs are being shown right alongside IR images and X-rays and the like. YAY! My takeaway is that RTI is an accepted part of the practice for art conservators. We are super happy to see that.
Finally, I was able to attend a 3-day workshop on RTI current practice and future directions, hosted by the Cyprus Institute in Nicosia, Cyprus in early June. This was “All RTI, All the Time!” and it was very productive to be part of a dedicated meeting like this. There were many noteworthy projects presented there, and I’ll point out a few. Up first, the work of Todd Hanneken and others integrating spectral imaging and RTI. While many of us have done spectral imaging and RTI going back several years, what is new here is the use of a MegaVision monochrome camera system, and taking 11 spectral bands. RTI was shot in the visible and used for luminance data to generate the RTIs, and the color data collected via the spectral bands was then applied to the images shot in visible light. The team also tested shooting full RTIs in each spectral band for comparison. In addition, work from the Visual Computing Lab in Pisa for a WebGL-based RTI viewer, released in January of this year, was discussed. At the moment, this work is limited to streaming large RTI files and then allowing the user to pan, zoom, and relight, but some possible future directions were outlined. It is open source, so you can try it out yourself. There is also a project at The University of Southampton to develop a web-based RTI viewer, and we received an update on that project, but I don’t have a link where I can send you just yet.
RTI group in the old city of Nicosia
As part of the discussion on where RTI is going, there was a lot of interest in quantitative uses of RGB and normals data. There are a variety of people working in this area, though they didn’t present directly at any of these meetings. Several of these folks are working with us at CHI, and we are very excited by this direction. First up, the work of Dale Kronkright, Greg Bearman, and several others to look at tracking changes through normals, and also to quantify normal calibration. You can find both papers here. Additionally, there is great work going on at Simon Fraser University under Professor Mark Drew to improve the accuracy of surface normals calculated from RTI data sets. They are also working on improving the appearance of RTI data in the viewing environment. There are a number of papers on this topic, most recently the masters thesis of Mingjing Zhang.
It is an exciting time to be working with RTI data! There is much more work going on than I could include here, and more that hasn’t been published yet. I want to close with a plug for the CHIForums where many topics like this are discussed, and there is a Projects Forum where folks can put up links to their work, as well as get comments and feedback.
Mark shooting photogrammetry sequence at Petroglyph National Monument
During the month of May we had the pleasure of doing more imaging work with Rock Art . This has included shooting some Reflectance Transformation Imaging (RTI) and some photogrammetry sequences at a couple of different sites. More importantly, we have had a chance to present some of this work to folks researching and recording rock art. We presented a workshop at theIFRAO conference in Albuquerque a couple of weeks ago. This was our first time attending the international conference. We were able to go to papers on all aspects of rock art research from all over the world. One of the things I love about rock art is that there are some things we just can’t know particularly about the older material where we don’t have living descendants from the culture to help us understand it. I find joy in that mystery.
One thing that is really clear is that rock art sites all over the planet are at risk and a lot of rock art is being lost every year. This is due to a wide range of factors from vandalism, to development, to earthquakes, to flooding and fires, to things as simple as natural rock falls. Part of our mission at CHI is to get tools to document these sites into the hands of folks who care about them. It is increasingly clear to me that teaching people how to capture the photographic image sequences that will allow the generation of full 3 D models, could really help us have records for future generations, and could provide a baseline of the current state of sites that would help monitor the changing conditions. The great thing about this approach is that it just takes a digital camera (and a monopod helps) and a bit of training about how to take the images in order to ensure complete coverage and high quality results. There are a number of different commercial packages available, and also some open source efforts, and the great thing is that how you capture the images is the same no matter where you want to process the images to get high quality 3D. We are now working on developing photogrammetry training for folks working in archaeology, historic sites, rock art sites, and related fields. Even if the data isn’t all processed in the short term, archiving the photographs that are properly collected will mean that anyone can create the 3D models in the future. To be clear, I don’t think anything takes the place of being in an actual site, and that it is critically important to protect and preserve these sites. But, given the fragility of these places, and also the inaccessibility of many of them, we should be gathering as much high quality data as we can as inexpensively as we can for ourselves and for future generations.
Setting up to shoot RTI of the 2 flute players panel at the Petroglyph National Monument, in Albuquerque, New Mexico.
June 8, 2011
Buzzed at AIC
At the recent annual meeting of the American Institute for Conservation (AIC) in Philadelphia, I was happy to see both past and future participants in the IMLS-sponsored CHI RTI training session program. I had to break the news to some interested parties that the sessions at the Worcester Art Museum and at the San Francisco Museum of Modern Art are already filled (with waiting lists) – but a few openings remain (apply here) in the sessions scheduled for the Smithsonian’s Museum Conservation Institute and the Indianapolis Museum of Art. Since AIC, several spots have already been taken.
Of course, conservators always want more visual information, and have been quick to understand the benefits of RTI for their work. Once conservators attend a CHI training session, RTI adoption spreads throughout the conservation community. At AIC, I heard several tales of conservators reaching out to their curatorial colleagues, presenting the extremely detailed technical art historical information to be gained from RTI. Curators were impressed! The ability to acquire this kind of detail is one of the hallmarks of RTI – in fact, in CHI’s Kress video Debbie Evans (paper conservator, Fine Arts Museums of San Francisco) mentions that RTI can allow the conservator to provide this important, detailed information to the curator. We look forward to more wonderful conservator/curator interactions!
By Elizabeth Peña
Just before the memorial day weekend Mark and I had the pleasure of attending the Society for the Preservation of Natural History Collections(SPNHC) conference. A national conference (and a few international folks too) held here in San Francisco this year. We gave an oral presentation and a demo camp presentation. Both were well received and afforded us the opportunity to speak to folks from various institutions. We have done a lot of work with fine arts institutions, but only a few proof of concept projects with natural history collections. It was a real treat to have a chance to go to some talks and panel discussions and also a collections tour at CalAcademy. I have always thought natural history museums were interesting and important and cool, but this conference really brought home to me just how important these collections really are as a scientific record of our planet. Collections large and small are critical to helping us answer questions about the natural world both now and in the future. A great example I saw on the collections tour were finches collected in 1905 – 1906 from the Galapagos, where the preserved skins allowed current researchers to do DNA analysis to answer new questions. Since DNA analysis didn’t exist when the specimens were collected, this clearly wasn’t something they were trying to enable. And yet, if we do a good job at collecting and preserving things (objects, data, specimens, etc.) then they just get richer and more valuable over time. That is one of our goals at CHI, to make data increase in value over time by collecting high quality imaging data and good records of how it was collected and what was done to it.
It was a great experience to be at SPNHC, and we hope to attend next year when it is hosted by the Peabody Museum at Yale.
A couple of weeks ago Carla Schroer conducted a workshop for a selected group of members of the Western Association for Art Conservation (WAAC) and members of cultural institutions in the Portland, OR area. The workshop was conducted at the beautifully restored and technologically advanced White Stag building of the University of Oregon, Portland. The title of the workshop was Digital Imaging Techniques for Conservation & Education. The three hour workshop was divided in two main sessions. In the first session, Carla lecture, via presentations, on the development and implementation of Reflectance Transformation Imaging (RTI) in the conservation field and cultural heritage institutions.
Carla in teaching mode.
Carla brought a small fossil with her to be photographed. She explained the minimum equipment requirements and recommendations for successful setups for different size and types of objects. Once all the mechanics where explained Carla, a volunteer from the group, and my self proceeded to capture the object with 24 images each with a different lighting position. She followed by explaining the postproduction steps to build the final RTI image as well as the use of available viewers for these images. She also demonstrated some of the computational enhancements that could be applied to the images and the amazing amount of information that can be obtained in this way. The day ended with a great Q&A session that really brought home many of the concepts that Carla had touch upon during the day finalizing a really informative workshop.
On a personal note I am very happy to have had the opportunity to assist Carla with the practical aspect of the workshop, but even more happy that Carla came to the meeting and that she was able to mingle and meet many of the members of this wonderful WAAC organization. Wishing a long lasting partnership and collaboration among the two institutions I remain thankful.
Yosi Pozeilov
Sr. Conservation Photographer
Conservation Center, LACMA
The Virutal Reality, Archaeology and Cultural Heritage (VAST) conference was great for us, and we are still enjoying being in Paris. This is a small focused conference with usually about 100 people in attendance. We put together a full day tutorial at the start of the conference with some of our favorite collaborators, and it was great to get this group together in person. We covered a lot of ground in a day with a nice mix of practical and theoretical material.
Carla setting up a demo during the tutorial
The rest of the conference was really good. I always enjoy the chance to catch up with various colleagues and friends, meet new folks working in this field, and see what other folks are up to. VAST is usually more on the computer science side of things, but there were a number of museum folks as well as archaeologists in attendance. There are always lively and sometimes controversial discussions, and this year lived up to that standard. Of great interest to me were the papers and discussions around how digital representations can track and reflect their true digital provenance from acquisition through to the finished process.
Mark and Carla in front of the Ecole du Louvre - site for the conference
I particularly appreciated a discussion with Holly Rushmeier from Yale University about what we mean by accuracy, high resolution, and quality in general? In particular what is even measurable, and how should we be measuring and recording it? Holly agreed that this is an area that needs some attention, and it will take the work of multiple institutions working together to come to any guidelines.
Having a chance to talk to Martin Doerr about mapping the digital provenance data to the CIDOC CRM always requires having your thinking cap on, but it’s so worth it. There were many other great people in attendance, interesting papers, and good food, wine and conversation. I look forward to next year when VAST is combined with VSMM (International Society on Virtual Systems and Multi-Media) in Alexandria, Egypt.
I really enjoy the opportunity to attend conferences and particularly the Computer Applications in Archaeology conference, which was in Granada Spain this year. It is a great opportunity to catch up with colleagues and friends, share experiences and get advice. This year was also an opportunity to sit down with our collaborators at Southampton University, Graeme Earl and team at the Archaeological Computing Research Group.
Graeme’s team recently won a significant UK grant to work with RTI technology and we are collaborating on the project. The team there has graciously allowed us to influence some of their priorities for enhancements to the RTI software as part of the grant work. We are still discussing some of the ideas, and it’s great fun to work with a group with such enthusiasm and desire to keep things moving forward for the growing community of RTI adopters.
I look forward to great things from this collaboration as it continues.
by Michael Ashley
I had the pleasure of being invited to a special congress in Seville this past week. Arqueologica 2.0 is the first international congress on archaeology and informatics held in Spain. The organizers managed to bring together over 200 participants from 17 countries representing over 100 organizations, to discuss and debate virtual archaeology and its role in archaeological practice. I found the congress to be inspirational, exceptionally well run, and mostly a whole lot of fun.
Participants included representatives from some of the most important organizations that work to document, understand, preserve and communicate cultural heritage around the world. In fact, the congress was a truly international gathering of professionals who are passionate about cultural history and memory.
So often, the discussions of computer graphics and archaeology focuses on the divide between technologists and practitioners of the discipline. I was impressed by the efforts of the congress program facilitators to get us to look beyond the fissure and to the mutual benefits of integrating visualization methods into archaeology.
Jane Crawford and Michael Ashley at the opening session at the site of Italica
Throughout the congress, many recommendations and suggestions were forwarded. I mention a few here because they resonate well and I hope we would generally agree are essential to promote the broad adoption of digital technologies in service of archaeology.
Bernie Frischer suggested that future Arqueologica meetings might have workshops to leave behind practical training with participants.
Richard Beacham, co-author of the London Charter on visualization in cultural heritage, called for case studies or pilot projects where we can test ideas and refine procedures in the real world.
Graeme Earl and several others suggested that virtual archaeology methodologies are actually archaeological techniques that must be truly integrated and carried out in archaeological practice.
Mediterranea Project
Many of us were kindly invited to listen to Alfredo Grande discuss the ambitious Mediterranea Project, which seeks to integrate the cultural heritage of the Mediterranean for research and public access and enjoyment. Core to the project is an attention to ‘big blocks’ that are essential to archaeological work practice – documentation and research, conservation and preservation, presentation and information.
One of the highlights of the scientific program was the uniquely organized quick fire plenary session that involved 23 of us lined up in the first row of the auditorium. From the stage, Víctor Manuel López-Menchero Bendicho gave us each <2min. to summarize our opinion on the state of virtual archaeology. This led to a lively debate with many more questions than answers, but there was general consensus that even now in 2009, we are only scratching the surface at the potential for digital archaeology.
Daniel Plentinckx in the spotlight during the rapid fire plenary
Arqueologica 2.0 was the first of what the sponsor organization, the Society for Spanish Virtual Archaeology (SEAV) (also launched at the congress), hope will become an annual event. I hope so, not because the world needs another conference, but because the spirit and enthusiasm expressed by our new friends in Spain is what the world needs.
My favorite quote from the congress was made by a colleague, “It’s more important who you work with than what you work on.” I would work with Alfredo and Victor anytime. I am sure we can find some really important things to do in an enjoyable fashion.
Muchas gracias por todo, nuevos amigos!
by Debra
There is so much going on at Cultural Heritage Imaging! In the next couple of weeks, Carla and Mark will attend the American Institute for Conservation’s (AIC) annual meeting and the ARARA conference. At AIC, Carla and Mark along with Philip Klausmeyer from Worcester Art Museum, will present a paper on “Reflectance Transformation Imaging: A New Conservation Tool for Examination and Documentation”. At ARARA, Carla and Mark will give a one-day workshop on Reflectance Transformation Imaging (RTI).
In June, Cultural Heritage Imaging is giving an on-site 4-day training class on Reflectance Transformation Imaging: Generating Digital Representations of Cultural Heritage Objects to staff members at the National Museum of the American Indian. And then there is the two-day RTI workshop, funded through a NCPTT grant, at the Presidio in July.
And much progress was made on the hardware and software for the IMLS project . But I’ll save those details for a future blog.
Cultural Heritage Imaging 