The incredible pace of digital transformation
Rembrandt, AI and the missing pieces of The Night Watch.
Rembrandt and AI. Now, I know, you are thinking: what in the world would Rembrandt have to do with AI? Today, in honor of Rembrandt’s birthday (July 15, 1606 in Leiden, The Netherlands), we are going to talk about just that.
Rembrandt and AI
Last September, I headed up to Amsterdam for one of my favorite tech events. (It was my first conference with “real live people” in almost 2 years.) One of the speakers was Robert Erdmann, Senior Scientist at the Rijksmuseum. He spoke about how he and his team “trained” AI neural networks to recognize certain pigment colors. They were taught to sort through millions of images of art from the collection, and even “paint” like Rembrandt. This work with the AI neural networks would be a significant factor in recreating the missing pieces of Rembrandt’s masterpiece, The Night Watch.
The missing pieces of The Night Watch
For those of you who don’t know the story of The Night Watch, it’s a good one. During the Dutch Golden Age, Rembrandt was one of the most popular (some say rebellious) painters of his time. In 1639, he received an important commission for a group portrait from Captain Banning Cocq and 17 other members of their civic guard — a local militia that defended Amsterdam mostly from the Spanish and kept all-around peace on the streets. Rembrandt was paid 1600 Dutch guilders which was a serious sum of money at the time. (This would be about 95,000 euro today. Quite the bargain in retrospect). Rembrandt finished The Night Watch in 1642. For the next 73 years, it happily hung in the banquet hall of the Kloveniersdoelen, the group’s guild house.
1715: The Night Watch doesn’t fit
In 1715, The Night Watch moved to the Amsterdam city hall. But…whoops, the giant canvas was too big to fit between two doors on the designated wall. The folks in charge decided to lop off a good 60cm chunk on the left side, some 7cm on the right, 22cm from the top, and 12cm from the bottom. Voilà, it fit just nicely. Unfortunately, it seems that nobody bothered to save the missing pieces. They completely disappeared.
Operation Night Watch
In the summer of 2019, the Rijksmuseum started Operation Night Watch, a grand-scale research and restoration project on the masterpiece. Centered around AI neural networks and ultra-high-resolution imaging, Operation Night Watch set a new standard in the world of art conservation techniques.
Enter big tech
To start, the team at the Rijksmuseum created a 44.8 gigapixel ‘base’ image. (You might think of it as a digital twin technology demonstrator, like our very popular Simcenter SimRod.) 528 different still photographs were stitched together digitally with the aid of AI neural networks. The AI networks recognized miniscule paint pixels and matched them precisely. From this, the team put the neural networks to work further on those missing pieces.
As Robert Erdmann explains in this movie, the team worked with digital copies of The Night Watch and a copy of the original painting by one of Rembrandt’s fellow painters, Gerrit Lundens. The AI neural networks were taught to recognize and correct perspective differences in the two paintings. Also, they could recognize the color scheme and project that onto the pixels of the missing pieces. As a next step, as Erdmann puts it, the neural network was “sent to art school” to learn how “to paint” like Rembrandt. By replicating the brushstrokes of the master, the team could make the best-possible digital twin of the missing pieces. And print the missing pieces for the general public to see.
An even better digital twin?
Although, the original painting with the digitally-printed missing pieces was only displayed briefly at the Rijksmuseum, it does show just how quickly digital transformation has taken on all aspects of daily life. Just 18 months later, the team undertook a crazy mission of capturing and stitching together an even-more detailed image. They used the 44.8 gigapixel base image as a guide to produce another even more detailed digital twin of The Night Watch. This time an ultra-high-resolution 717 gigapixel image – the largest and most detailed photograph of any artwork to date.
Go ahead: dream bigger
And you can experience it online. This image is four times sharper than its predecessor (the 44.8 gigapixel base image) and provided enough data so that researchers, conservation experts, and art historians could zoom into pinhead-shaped particles of Rembrandt’s pigments.
The more data, the better
Thanks to the rapid-pace of digital transformation and smart AI-driven technology, the Operation Night Watch team (and those AI neural networks) now have even more data to work with. With all this information at the digital-ready, the restoration team at the Rijksmuseum can make the best-informed decisions regarding future conservation plans.
As we move further into the new world of digital ecosystems, technology advancements like Operation Night Watch will show us the positive power of digital transformation for humanity as a whole.
Now, if they could only teach those AI neural networks to find those missing pieces from 1715…
_____________________________________________________________________________________________
And speaking of AI, here are just a few of the latest Simcenter AI resources:
Images courtesy of The Rijksmuseum.
