Category: Research update

Dealing with multidimensional uncertainty: The house of the painter Gillis van Coninxloo

Chiara Piccoli and Weixuan Li

A crucial question in our project is how to identify the location of a house in the urban space. As we know, archival sources record addresses at the street level, sometimes with additional information regarding their relative location (e.g., at a corner with another street or in front of a recognizable building). This way of spatially “mapping” the city in early modern sources poses challenges to our possibilities to securely pinpoint a house on the map. On the 2D level, uncertainties in locations are expressed through fuzziness and various degrees of transparency of a possible area so that the range of uncertainties involved in the 2D representations can be communicated explicitly.[1]

When it comes to the 3D level, however, the additional dimension increases the uncertainties exponentially as all details need to abide by the stringencies of the exact house location and its spatial boundaries. This problem has not manifested itself in the case study of the house of Pieter de Graeff at Herengracht 573. There, the house is still standing and the memory of its owners has been preserved. But what to do when the early modern house is long gone and the owners have been forgotten with the passing of time? Can digital mapping tools and 3D reconstructions help us to gain some insights in how the domestic space was used and experienced also in this case? Often, the evidence is so scarce that there is no ground to even start trying to create a reconstruction hypothesis; but what about in other situations, for example when the street is known, but the precise location of the house can only be hypothesized? We are going to explore these questions with the house of the painter Gillis van Coninxloo.

Coninxloo: Life and Work

Gillis van Coninxloo (Antwerp, 1544 – Amsterdam, 1607) was one of the most important and celebrated painters in Amsterdam at the turn of the 17th century and had a profound influence on the stylistic development in landscape paintings in the Dutch Republic (Fig. 1). Born into a family of painters, Coninxloo apprenticed in Antwerp and established his painting career there for over a decade until the religious strife in the Southern Netherlands under the Spanish rule forced the Reformed artist to flee north in 1585. Coninxloo left first for Middelburg and then in 1587 for Frankenthal in Germany before permanently settling down in Amsterdam in 1595, where he died in 1607.[2]

As one of the few established painters emigrating from the South who resumed their career as ‘kunstschilders’ (artist painters) in Amsterdam, Gillis van Coninxloo introduced a new approach to Flemish landscape painting.[3] His up-close landscapes turned forests into the main subject instead of the backdrop by crafting elaborate compositions of trees and fine shades of the leaves on scale (Fig. 2). His dense, airless forest scenes influenced and inspired the younger generation of landscape painters of the 1600s in Amsterdam, such as Roeland Savery and David Vinckboons, in addition to his numerous pupils. In 1604, art theorist Karel van Mander wrote in his Schilder-boeck that ‘[Coninxloo] is, as far as I know, the best landscape painter of his time; his style is now frequently imitated in Holland’.[4] Besides his influence in the stylistic development in landscapes, Coninxloo was, too, active in the art trade.[5]

After his death in 1607, the auction of his estate gathered almost the entire artists’ community in Amsterdam. A probate inventory of Coninxloo’s possessions in his house prior to the auction survives and offers a valuable and rare example of how this renowned painter (who was also active in the art market) arranged his workshop and operated his business in the domestic space in the first decade of the 17th century.

Fig.1 Andries Jacobsz Stock, Portrait of Gillis van Coninxloo, 1610, book illustration published by Hendrik Hondius I.

Fig. 2 Gillis van Coninxloo, Forest Landscape, 1598, oil on panel, 42 x 61 cm, Vienna: Liechtenstein Collection, Inv. GE751.

Coninxloo’s inventory (Fig. 3) is preserved at the Amsterdam City Archives[6] and it is part of the sample that Weixuan Li is analysing for her study on the creative communities in Amsterdam. A large number of paintings including many unfinished ones and a significant amount of art supplies probably suggested that the artist not only produced paintings but also ran a shop at home. Interestingly, the inventory speaks of both ‘Coninxloos winckel’ and an ‘achter winckel’ on the first floor alluding to the possibility that the studio and painting shop were located in the same space. This aspect has made this house an interesting case study to see what kind of contribution could 3D modelling give to our understanding of its internal room organization, notwithstanding the uncertainty about its location as will be further discussed in the next section.

Fig. 3 Two pages from the inventory of Coninxloo’s house. The list of objects starts on fol. 69r with the entrance hall (‘voorhuis’).

Sources for locating Coninxloo’s house

The presence of the painter Gillis van Coninxloo in Amsterdam is registered for the first time in the ‘Poorterboeken’ on 22 April 1597.[7] On 23 August 1603, Coninxloo posted banns with Geertje van Eede.[8] In this document, he appears for the first time as living on the Turfmarkt, and the addition of ‘8 ans’ in smaller caps on top of ‘wonende’ indicates that he had been living in Amsterdam for eight years. As usual, no additional indication of a more precise location is given to pinpoint the house where Coninxloo lived.

Traces in the archive

The starting point to identify the range of possible candidates on the Turfmarkt is the reference contained in his probate inventory to a debt of 200 gulden he had with Arent ten Grootenhuys, one of the founders of the VOC, for one-year rent of a house. Ten Grootenhuys was the grandchild of the merchant Arent Hudde, who we find in the Kohier of 1562[9] as the owner of three houses on the Turfmarkt, and again in the Kohier of 1568/69,[10] this time mentioned as the owner of two more, thus owning a row of five adjacent houses. The Kohier of 1578[11] provides us with the picture of the situation ten years later: after the list of houses belonging to, and rented out by, the Nieuwe Nonnen convent, we find Hudde clearly stated again as the owner of five houses, all rented out. The names of the tenants are given and the fifth one is the painter Dirck Barentsz. Barentsz appears therefore to have rented the last (most south) of Hudde’s five houses.

Additional pieces of information are contained in the execution of will of Cathrijn Jacobs, Hudde’s widow, dated 1 September 1600.[12] There, we find out that she left her possessions, among which the houses on the Turfmarkt, to her grandchildren. Specifically, the “children of Reijmerich Hudde” (among them, Arent ten Grootenhuys) inherited the house where the painter Pieter Isaacsz used to live. No relative location of this house in relation to the others is explicitly mentioned in this document, but its higher value could be taken as an indication of a bigger house size. Another key to the puzzle comes from an archival document indicating that in 1607 Arent ten Grootenhuys sold a house on the Turfmarkt to the painter Frans Badens.[13] Some of the characteristics of this house are described in this source, such as that it had a shared alley with its southern neighbor.

The years in which Coninxloo lived on the Turfmarkt (1603-1607) fall in between the periods covered by the archival sources and therefore Coninxloo is not mentioned in any of them. Based on the previously discussed evidence, Bas Dudok van Heel and Marten Jan Bok made the logical step to assume that the four painters (Barentsz, Isaacsz, Coninxloo and Badens) inhabited in sequence in the same house – the fifth of the houses originally built by Arent Hudde.[14] It seems indeed straightforward to make a one-to-one connection between the house inherited by the children of Reijmerich Hudde, and the house for which Coninxloo still needed to pay Ten Grootenhuys one year of rent. The archival research conducted in the context of our project shows however a less clear situation:[15] By 1615 the children of Reijmerich Hudde seem to own more houses on the Turfmarkt, instead of only the one that is indicated in Cathrijn Jacobs’ will. The fact that Badens bought the house from Ten Grootenhuys only a few months after the auction of Coninxloo’s belongings can be taken as an indicator that he indeed acquired the house where Coninxloo lived, which had become vacant. The likelihood that Badens and Coninxloo inhabited the same house is therefore high, but the possibility that they lived in different houses cannot be excluded with certainty at the moment. More research is needed to follow through the questions opened up by the archival documents and to understand whether they have any implications for locating Coninxloo’s house.

Archaeological evidence

The archaeological excavations at Oude Turfmarkt 135-139 conducted by the Amsterdam Monuments and Archaeology Office in 2005 helped to shed more light on the location of Badens’ house.[16] Their documentation allows us to see the foundations of three parcels built in the first quarter of the 16th century and of a 1m-wide alley, in between parcels II and III (Fig. 4). As already suggested by Bok, for its characteristics, parcel II corresponds most certainly to the house that Badens bought in 1607 from Ten Grootenhuys.[17] This house was located on a parcel in the area now occupied by the Special Collections and the Allard Pierson Museum Cafè (n. 139) (Fig. 5).

The heavy foundations on which the houses were built are exceptional for residential buildings. Their sturdiness was given by the necessity to prevent them from sinking into the unstable terrain. The archaeological investigations showed that they were in use for a long period of time, thanks indeed to their solid construction. We are therefore looking at the same perimeter that the houses had in the early 17th century. The excavated remains clearly show that the parcels become longer towards the south-east (as it is also visible in the 1625 map by Balthasar Florisz van Berckenrode, see Fig. 6). This difference in length originates from the necessity to negotiate the available space between the river’s bank on one side and the arch-shaped row of houses of the earlier Nieuwe Nonnen convent which followed the river’s course on the other.

Fig. 4 Archaeological remains documented during the excavations conducted in 2005 by the Amsterdam Monuments and Archaeology Office (Gawronski et al. 2010, p. 19).
Fig. 5 GIS mapping of georeferenced aerial images, historical maps and excavation plans with the supposed perimeter of Frans Badens’ house highlighted in blue (C. Piccoli).
Fig. 6 The portion of the map made by Balthasar Florisz van Berckenrode in 1625. The houses corresponding to the area uncovered by the 2005 archaeological excavations are highlighted.

3D modelling as a drafting and exploratory tool

By inference, as discussed above, Badens’ house is at present also our most plausible candidate for the house where Coninxloo lived. As we showed, however, the evidence at our disposal is insufficient to exclude other possible candidates. This is the first dimension of uncertainty that we are dealing with when we are trying to position Coninxloo’s house on the map. An additional dimension of uncertainty comes from the internal arrangement of space. In the case of De Graeff, although his house has been modified over the century, it still offered some anchor points to compare and relate which rooms were mentioned in the inventory.[18] There, the sources of uncertainty were related to the relative position of objects within the rooms and their appearance. In Coninxloo’s case, the inventory is organized by room but we do not have any extant remains that help us with this task. The only information about its dimension is the (incomplete) perimeter of the house supposedly owned by Badens that emerged from the archaeological excavations displayed in Fig. 5.

Given these circumstances, we use 3D modelling as an exploratory and drafting tool that allows us to visualize our hypotheses and interpretations. The 3D reconstruction acts as a platform for discussion and is presented in a visual language that conveys the highly hypothetical and exploratory nature of this work. The rendering method is therefore not photorealistic but schematic and wireframe (Fig. 7). What 3D modelling can contribute to in dealing with multidimensional uncertainty, for example, is the possibility to visualize what the house could look like in the various possible scenarios. We can explore questions such as to what extent the changes in the available interior space matter when the same room partition is applied to the progressively smaller parcels.

One of the possible outcomes could be the observation that even with different parcels, the changes in the available inner volume would not be significant after all. If a smaller size would have little impact on how space could be used inside, then the problem of the secure identification of the house location would become less important in view of what we are interested in investigating. Taking it further, this exploratory use of 3D modelling could also help us to support the hypothesis that all the above-mentioned painters lived in the same house in case we would be able to demonstrate that it offered particularly favorable conditions, for example in term of lighting and access, for the installation of an artist workshop.

Fig. 7 An initial hypothesis of Coninxloo’s house visualized in a wireframe 3D model, superimposed on top of the results of the archaeological excavations in the area published in Gawronski et al. 2010 (3D visualization by C. Piccoli). The rooms’ names correspond to those mentioned in Coninxloo’s inventory.

[1] See Weixuan Li’s forthcoming dissertation: Artists and the Creative Urban Space: Deep-Mapping Painters’ Locations in Golden Age Amsterdam (University of Amsterdam).

[2] J. Briels (1997). Vlaamse schilders en de dageraad van Hollands Gouden Eeuw, 1585-1630: met biografieën als bijlage, Antwerpen: Mercatorfonds.

[3] Many immigrants from the Southern Netherlands who were known as painters did not have any known work and therefore were questioned for their artistic qualifications. Only Coninxloo, Hans Bol, Jacob Savery as established Flemish landscape painters continued to paint in Amsterdam. Most landscape painters with roots in the South emigrated with their parents at a young age and were trained in the northern cities like Amsterdam. D. Van der Linden (2015). ‘Coping with Crisis. Career Strategies of Antwerp Painters after 1585.’ De Zeventiende Eeuw. Cultuur in de Nederlanden in Interdisciplinair Perspectief 31 (1), pp. 18–54. E.J. Sluijter (2015). ‘Career Choices of Migrant Artists between Amsterdam and Antwerp. The Van Nieulandt Brothers.’ De Zeventiende Eeuw. Cultuur in de Nederlanden in Interdisciplinair Perspectief 31 (1), pp. 101–37.

[4] ‘[…] soo weet ick dees tijt geen beter Landtschap-maker: en sie, dat in Hollandt zijn handelinghe seer begint naeghevolght te worden […]’. Van Mander, Karel, Het Schilder-boeck, 1604, fol. 268r.

[5] J.M. Montias (2002). Art at Auction in 17th Century Amsterdam. Amsterdam: Amsterdam University Press, pp. 115-119.

[6] SAA, Inventaris van het Archief van de Notarissen ter Standplaats Amsterdam (5075), notary Frederik van Banchem (nr. 12), inv. nr. 262, fols. 68-88 (11 January 1611).

[7] SAA, Inventaris van het Archief van de Burgemeesters: poorterboeken (5033), inv. nr. 1, fol. 183v (p. 354).

[8] SAA, Inventaris van het Archief van de Burgerlijke Stand (5001), Huwelijksregisters, inv. nr. 411 (23 August 1603), p. 8.

[9] NA, Archief van de Staten van Holland voor 1572, inv. nr. 1206, kohier van de tiende penning te Amsterdam 1562, fol. 313r.

[10] SAA, Inventaris van het Archief van Burgemeesters: stukken betreffende verscheidene onderwerpen (5028), inv. nr. 550: Kohier van de verhuring van de huizen van fugitieven, geconfisceert in Amsterdam, 1568-1569, 1 deel, S III, p. 39, nr. 17, fol. 26.

[11] SAA, Inventaris van het Archief van Burgemeesters: stukken betreffende verscheidene onderwerpen (5028), inv. nr. 661: Register van de 100ste penning van de onroerende goederen, gelegen aan de Oude Zijde in Amsterdam, 1578, 1 deel, S III p. 39, nr. 18, fol. 189.

[12] SAA, Inventaris van het Archief van de Notarissen ter Standplaats Amsterdam (5075), notary Salomon Henrix (nr. 4), inv. nr. 11 (Band 3), 2v ff. (1 September 1600).

[13] SAA, Archief van de Schepenen, Register van Schepenkennissen (5063), inv. nr. 11, fol. 285 (15 June 1607). This document is cited by Dudok van Heel as a ‘Rentebrief’ (B. Dudok van Heel (1987). ‘Enkele adressen van zestiende-eeuwse kunstschilders’, Amstelodamum 74, pp. 1-7 (7)). We thank Marten Jan Bok for having retrieved the document and shared the correct reference with us.

[14] Dudok van Heel 1987 and M.J. Bok (2005). ‘Een Ganymedes van Francois Badens en de werkplaats voor schilderijen in Italiaanse stijl aan de Oude Turfmarkt’, Amstelodamum 92 (4), pp. 3-14.

[15] We thank Bart Reuvekamp for his help with this archival research and for the transcription of Coninxloo’s inventory, and Judith Brouwer for checking it.

[16] J. Gawronski, R. Jayasena, J. Veerkamp (2010). Van Amstelbocht tot Binnengasthuis Archeologische opgraving Oude Turfmarkt, Amsterdam (2005)’, AAR (Amsterdamse Archeologische Rapporten) 31, 2010. I thank R. Jayasena for having shared the original AutoCAD drawings of the excavations with me (C.P.).

[17] Bok 2005, p. 5.

[18] C. Piccoli (forthcoming). ‘Home-making in 17th century Amsterdam: A 3D reconstruction to investigate visual cues in the entrance hall of Pieter de Graeff (1638-1707)’, in G. Landeschi and E. Betts (eds.), Capturing the senses: digital methods for sensory archaeologies, Springer.


Analyze & Experience: Towards a Research Environment for 3D Reconstructions

How to design & evaluate user interfaces as well as interaction paradigms for accessing 2D & 3D datasets involving uncertainty?

This was the question which I addressed in my project stream within Virtual Interiors. My work involved designing, creating and evaluating a demonstrator for a 3D research environment, in a co-design setting with Chiara Piccoli and during embedded research at project partners Netherlands Institute for Sound & Vision and Brill Publishers.

This post summarizes the work pertaining the 3D research environment, points to additional resources and also looks ahead to future opportunities.

1. The challenging task of 3D interface design

It is not straightforward to create user interfaces for accessing 3D reconstructions in a historical setting. User interfaces can be seen as an “in-between device” between the historically interested user and the actual 3D reconstructions and underlying data (see e.g. [1]).

The interaction with 3D environment and assets is challenging in itself and navigation issues and disorientation are commonplace [2]. This affects the usability of a prospective interface [3]. In addition, the usability is influenced by the number and complexity of interface features: the paradox is that more features might make an interface more useful to a researcher, but also harder to use. Striking a balance between functionality and usability is therefore of key importance.

Achieving this balance is also more arduous because there is a lack of documentation of best practices for interface design in a 3D context. Hence, it is hard to determine which “standard” set of features is necessary for interacting with 3D historical content [4].

2. Exploring requirements

To shed more light on these issues, I spent the first months of my work as a postdoc in the Virtual Interiors project (2019) in exploring the requirements for a 3D research environment. This was done via an analysis of previous use cases for 3D, design thinking [5] sessions with the other project members. Some requirements for a 3D research environment that came up were the ability to view the 3D space from different perspectives, to interact with objects, to add analytical layers (uncertainty & lighting), to annotate and to view context information. Furthermore, a user study with digital humanists was conducted [6].

From this user study, it became apparent that historical and spatial context are of key importance for a 3D research environment. From a thematic analysis of session transcripts, we identified the spatial context dimensions object properties, object relationships, and perception of space, pictured in Figure 1.

Figure 1. Spatial context dimensions from Huurdeman & Piccoli, 2020
(● object properties, object relationships & perception of space)

More information:
📖 Hugo Huurdeman & Chiara Piccoli (2020), The Importance of Context in Search User Interfaces for 3D Content (Proceedings CHIIR 2020)

3. The Virtual Interiors 3D research environment

Based on the initial phase of analyzing requirements, a prototype research environment was designed and developed in an iterative way. The prototype addressed the requirements mentioned above. While a full discussion of its functionality, the rationale behind it, the technical structure and the reconstruction process is available elsewhere [16], I discuss some key features here.

Analytical & Experiential user interfaces

Historical 3D environments can be utilized by widely different audiences, ranging from the casually interested to topical experts. For these audiences, different user interface features can be worthwhile. Instead of creating one interface to cater for all of their needs, which may lead to to the aforementioned tensions in the balance between functionality and usability, I decided for a different approach.

To cater for these different audiences and their goals, we constructed a “multi-layer interface” [7], in which two distinct interfaces use the same underlying framework and data. On the one hand, there is an analytical interface layer, aimed at researchers, and on the other hand a more experiential interface layer, which can be used by both researchers and other historically interested individuals:

  • The Analytical interface allows researchers to take a deep-dive into underlying research data about domestic interiors, view uncertainty overlays, explore associated archive documents and to follow leads to connected Linked Data for instance examining artist biographies, related artworks and various historical data sources. Moreover, it is possible to explore hypotheses related to object arrangement and lighting. These analytical interfaces are best experienced via desktop computers or tablets (see Figure 2A & 2B).
  • The Experiential interface makes it possible to experience historical spaces intuitively, by e.g. utilizing Virtual Reality and the motion sensors of mobile phones (see figure 2C & 2D). Casual users and researchers can “jump into” a 3D historical space, look around and interact with the objects around them, in more immersive [8]. Experiential views have been explored in the context of Chiara Piccoli’s 3D reconstructions of domestic interiors, but also during my embedded research at the Netherlands Institute for Sound & Vision.

Read more about analytical interfaces:
📖 Hugo Huurdeman, Chiara Piccoli (2021), 3D Reconstructions as Research Hubs (Open Archaeology journal)
Read more about experiential interfaces:
📖 Hugo Huurdeman, Jesse de Vos (2020). Visualizing a Radio of the Past using Technology of the Future (blogpost, Neth. Inst. for Sound & Vision, R&D blog, with link to early 3D demo)

Integrating Linked Data

Another key differentiation point between the Virtual Interiors research environment and other 3D frameworks [9] is the pivotal role of Linked Data [10] in enriching our content. Linked Data provides a common approach to more easily interconnect data, and has also been used in the context of (digital) history [11]. More scarcely, however, it has been applied to the context of 3D reconstructions [12].

For the Virtual Interiors 3D reconstructions, we use Linked Data to dynamically integrate biographical information from Ecartico, but also additional (and visual) resources pertaining the reconstructions at hand. For instance, selecting Quellinus’ portraits in the entrance hall of the reconstructed house of Pieter de Graeff and Jacoba Bicker reveals a gallery of works by the same creator, and works of a similar type (from Adamlink and Wikidata). Linked Data can have added value for these kinds of interfaces, although the involved SPARQL [13] queries need to be designed carefully, and it is very important to clearly display provenance.

Figure 5: impression of Linked Data functionality in analytical interface (prototype June ’21)

The connection between Linked Data and both experiential and analytical interfaces is also illustrated by Figure 5 and 6. The analytical interface shows biographies, links and galleries based on Linked Data in the sidebar, while the experiential interface displays image galleries via a spatial projection surrounding and immersing the user.

Figure 6. Impression of the spatial projection of Linked Data in experiential interface (prototype June ’21)

Read more:
📖
Hugo Huurdeman, Chiara Piccoli, Leon van Wissen (2021), Linked Data in a 3D Context (DH Benelux 2021).

Analytical searching

Another important element of our research environment demonstrator is the ability to do anlaytical searching. As indicated in [6], 3D scenes and reconstructions can potentially consists of a vast number of objects and details, which may be hard to explore due to issues in navigating 3D spaces. Using the demonstrator’s approach, it is possible to browse a textual list of objects in the scene, but also to highlight singular as well as categories of objects — while keeping the context of the objects surrounding it. For instance, a user might want to see all furniture in the the entrance hall of Pieter de Graeff’s and Jacoba Bicker’s house (Figure 5). These elements are highlighted, while the surrounding objects are still (dimly) kept visible.

4. Evaluation

As mentioned, the design and creation of the research environment took place in a co-design setting with Chiara Piccoli, but also involved frequent feedback from the other team members. Besides this internal feedback, also external feedback is of key importance to resolve usability and system issues. A more formal summative study of the analytical desktop prototype, combined with Chiara’s 3D reconstructions and research about Pieter de Graeff’s entrance hall, was conducted in June 2021 (6 participants).

Participants in the study filled out the System Usability Scale (SUS) questionnaire, which measures usability. The resulting SUS score was 77,9. This means that the usability is clearly above average, with only a need for minor improvements.

The majority of minor issues coming to light in the study could be fixed in the subsequent weeks, which included:

  • Navigation issues. The familiarity with 3D environments of the participants influenced how easy it was to navigate the scene. We resolved this by providing more explicit instructions, and giving a higher prominence of a guided tour which informs about the possibilities of the 3D environment and viewer.
  • Usability issues — these issues could be resolved rather easily, and were for instance related to the selection and deselection of objects in the scene, the textual labels of features and buttons.
  • Technical issues and bugs — the majority of these smaller issues could also be fixed. More substantial issues, such as a lower performance on old computers could be partially alleviated by adding settings for simplified display and custom framerates.

5. Preservation strategy

Generally, cutting-edge 3D technologies are of an unstable nature. Software and platforms continuously evolve, often leaving behind support for prior technologies. This might render apps from the past non-functional [14, 15].

To mitigate this issues, the following approach was taken in Virtual Interiors:

  1. to use only open tools for creating the 3D models (e.g. Blender),
  2. to utilize open file formats (e.g. CSV, GLTF), and
  3. to use open technologies for creating our research environment (e.g. BabylonJS).

A key component is the open-source 3D game engine for the web named BabylonJS. We chose a web technology to provide us with the flexibility to support various platforms (ranging from mobile to desktop) and to be able to use more immersive modalities such as VR via the emerging WebXR standard.

In our turn, we are also publishing the underlying data and tools via an open-source tools license as well, and make these accessible in Zenodo and DANS Easy by the end of the project. Documentation of process, demos and code is also a key focus, augmented by screencasts and screenshots of the demonstrator at different points in time.

6. Conclusion

Despite the end of my tenure at Virtual Interiors, the developments do not end here. Additional 3D reconstructions will be integrated into the research environment. Moreover, the 3D research environment will be refactored and modularized by Saan Rashid, who recently started as a software engineer in Virtual Interiors.

This blogpost discussed the creation of a demonstrator for a 3D research environment. I have outlined the challenges, our approach to tackle them, but also the key importance of evaluation when creating these kinds of novel interfaces. I highlighted some of the numerous exciting opportunities to use 3D reconstructions in a research context, ultimately providing a form of “analytic immersion”.

Try out the experimental prototype (desktop version) with the entrance hall of the 17th-C house of Pieter de Graeff & Jacoba Bicker, reconstructed in 3D by Chiara Piccoli (prototype version: August 2021).

Footnotes*

  1. Gane & Beer (2008)
  2. see e.g. Chittaro & Ieronutti (2004); LaViola et al (2017); Jerald (2016)
  3. See e.g. Jacko (2012); Lewis (2012); Mayhew (1999)
  4. See for instance Champion (2019); Wood et al (2019)
  5. See e.g. https://www.interaction-design.org/literature/topics/design-thinking
  6. See Huurdeman & Piccoli (2020) and this summary blogpost.
  7. See Shneiderman (2002).
  8. E.g. Dede (2009), Immersive Interfaces for Engagement and Learning.
  9. See e.g. the survey of features in current 3D frameworks by Champion & Ramahan (2020).
  10. See e.g. Antoniou & Van Harmelen (2008)
  11. see e.g. De Boer et al. (2016), Linked Data for Digital History; Meroño-Peñuela et al. (2020), Ontologies in CLARIAH; Zamborlini et al. (2017), Toward a Core Conceptual Model for (Im)material Cultural Heritage.
  12. As e.g. done by Kuroczyński et al. (2016)
  13. SPARQL is a query language commonly used for Linked Data repositories
  14. See e.g. Delve et al., 2012, The Preservation of Complex Objects—Volume 1; Lischer-Katz et al., 2017, Supporting Virtual Reality and 3D in Academic Libraries. Also, projects such as Pure3D are looking at infrastructures for 3D visualizations.
  15. More about this topic will be discussed in Huurdeman, Posthumus, Van den Heuvel, Beyond Dynamic Drawings: Restoring and Re-using Interactive 3D Visualizations (in preparation).
  16. Huurdeman & Piccoli (2021), 3D Reconstructions as Research Hubs

Abbreviated citations correspond with Huurdeman & Piccoli (2021), 3D Reconstructions as Research Hubs, full details in paper’s reference list.

Private libraries as a microcosmos of knowledge: Travelling through time and space with Pieter de Graeff’s book collection

L’une des meilleures manières de recréer la pensée d’un homme: reconstituer sa bibliothèque
-Marguerite Yourcenar

If we were to select only one feature typifying the 17th century Dutch Republic, we would certainly need to single out its thriving book culture. A cultural industry that was the largest in Europe at that time (see e.g. Dijstelberge and Verkruijsse 2010, 143; Hoftijzer 2015), the Dutch book sector produced a higher number and had a more internationally oriented market than the painting industry (Pettegree and der Weduwen 2019, 2). According to the estimated figures discussed in Pettegree and der Weduwen, paintings on the walls of Dutch homes would collectively amount to about 3 million pieces, while the book production would have reached as many as 300 million, with at least 4 million books traded at auctions (Pettegree and der Weduwen 2019, 1).

Within my postdoc project on Visualizing Amsterdam Interiors, the case study of Pieter de Graeff has offered us the opportunity to investigate the presence of this important product of the Dutch cultural industries at his house on the Herengracht 573. From the probate inventory of his properties, which was drawn up after his death in 1707, we know in fact that his home was equipped with a library (‘boeken kamer’). Thanks to a reference to this room in one of the passages of his almanacs, we can also pinpoint its location in the house with certainty, namely in the room above the entrance hall. The few items listed in the inventory point towards a simple furnishing, as we would expect for rooms of this kind, and an additional note written in smaller caps at the end of this section of the inventory reminds that a more specific list is to be made of the books that are present in this room and elsewhere in the house. Although this more detailed registration of books doesn’t seem to have survived, one copy of the De Graeff’s book auction catalogue is currently preserved at the National Library of Russia in St. Petersburg (fig. 1). Its digital copy is included in the Brill Book Sales Catalogues Online (BSCO), which gives access to thousands of facsimiles of book sales catalogues printed in the Dutch Republic before 1801. Until now overlooked in literature, this book catalogue comprises more than 2,300 titles, in several languages and covering a variety of subjects.

The title page of Pieter de Graeff’s book auction catalogue
Fig. 1: The title page of Pieter de Graeff’s book auction catalogue.

Reconstructing De Graeff’s book collection

Bart Reuvekamp and I have prepared an annotated transcription of the book catalogue, which is facilitating the quantitative and qualitative analysis of its content. We are also planning to make it available to other researchers through the DANS repository, in order to foster broader comparisons and quantifications across different book auction catalogues. With the collaboration of Leon van Wissen from the Golden Agents project, we aim in fact to further enrich this transcription with links to e.g. Wikidata and the Dutch National Thesaurus for Author Names (NTA). To this end, we added the corresponding VIAF identifier to each author in the catalogue, which both overcomes ambiguities in their identification (e.g. due to different spellings of an author’s name), and enables us to connect them to other resources. Titles will be moreover matched with the datasets of the STCN and Worldcat, which, besides offering an interesting technical challenge, will give us an idea about the amount and distribution of existing copies.

As well expressed by Marguerite Yourcenar in the quote at the beginning of this post, one of the best ways to recreate what one person thinks is to reconstruct their library. The books in the auction catalogue indeed clearly show an extensive attention for juridical and theological matters, and point to specific interests in historical events and geopolitical issues, as well as being indicative of the De Graeff’s political sympathies (a detailed analysis is in preparation). There are moreover several descriptions of faraway places, travel guides, manuals to learn foreign languages, and dictionaries. De Graeff’s library, encapsulating the 17th century microcosmos of knowledge, allowed travelling through time and places, exploring different locations and historical periods, and thus connected this small room in a house along one of the Amsterdam canals and its inhabitants with the broader international context.

Whomever works with book auction catalogues as primary sources is however confronted with the question as to what extent they are really representative of the book collection owned by the person advertised on their title page. For example, cases are known of publishers who, eager to sell their own stocks, included them in somebody else’s book catalogue (Van Eeghen 1978, 246). Were all the books in this catalogue really owned by Pieter de Graeff? And if so, how did they enter his collection? In addition, what other books did he own besides those that are listed? Thanks to the information we derived from archival documents and the De Graeff’s almanacs, we have sufficient evidence to shed light at least partially on these questions: We can not only prove that he kept books originally owned by several of his family members, but also clarify the circumstances under which some of the books in the book catalogue entered his collection. Through these documents, moreover, we are also able to pinpoint additional books that are not present in the catalogue, but that were in de Graeff’s hands at some point during his life, for example volumes that he lent to other family members or close acquaintances.

Reconstructing the physical space: De Graeff’s ‘boeken kamer’ at Herengracht 573

Besides reconstructing the content of De Graeff’s library, the other complementary aspect of this research was to reconstruct the physical space where (at least some of) these books were kept in his house. As already mentioned in a previous post, none would guess that a library existed at this location in the 17th century, given the numerous modifications that this space underwent over the course of several generations, and its different uses for which it has been destined (from domestic to exhibition space, as shown in fig. 2). The aim of the 3D reconstruction hypothesis (fig. 3) was therefore first of all to help visualize the original function of this area. In addition, it provided a way to suggest a rough estimate of the number of volumes that could be kept in there, thus giving us also an idea about how many would have been stored elsewhere in the house. The starting point has been the collection of evidence of similar contemporary libraries and study rooms depicted in paintings and engravings, as well as the comparison with the Thysiana library, a slightly earlier, still preserved, example in Leiden.[1] The Thysiana has been especially useful to check the thickness of the shelves and the number of books on each shelf, which are arranged according to their format.

Fig. 2: The appearance of one side of what used to be De Graeff’s library, transformed in exhibition space for the now closed Museum of Bags and Purses and completely unrecognizable in its original function.
Fig. 3: Current reconstruction hypothesis of Pieter de Graeff’s library, originally located on the second floor of his house at Herengracht 573.

As in the case of the reconstruction of the entrance hall (‘voorhuys’), ‘intellectual transparency’ in the reconstruction process plays an important role in this research (see Piccoli, forthcoming). In this case, besides a color-coded visualization of uncertainty, this issue is dealt with by proposing alternative reconstruction hypotheses (e.g., of the position of the closet, which I cannot derive with certainty from the inventoried list of objects in this room, and of the appearance of the shelves, which in one reconstruction is based on those in the Thysiana, while in another is rendered with a simpler wooden frame as often depicted in other visual sources). The thesis project of Alessandro Pantò, a student of the MSc in Computer Science at the Leiden University (LIACS), will allow us to explore the potential of the game engine ‘Unreal’ to create a VR-based walk-through of this room, which takes into consideration also how to convey the uncertainty in the visualization.[2]

Behind the scenes of the 3D modelling process: strategies to create the books in the library

One of the modelling challenges posed by this room was the need to create all the books that filled up the shelves: modelling them one by one in detail would be not only too time consuming, but would also lead to an excessively high polygon count, which can be problematic when using the model for real time interaction. I therefore modelled the rows of books as two-sided simple geometries and relied solely on the pictures taken at the Thysiana of their spines to give the impression of their details and volumetric properties. Figure 4 illustrates the result of this technique.  

Fig. 4: The low-polygon books that fill up the shelves are modelled (in Blender) as surfaces and not as complete 3D objects, thus decreasing the overall polygon count without compromising the results.

For the books I put on top of the closet, near the window, a more detailed geometrical representation was needed. For this reason, I used another technique called ‘Structure from Motion’ (SfM), which allows the estimation and reconstruction of the 3D geometry of an object from a series of pictures. To this end, I took several overlapping pictures of books from different perspectives, and used them as input in the open source photogrammetry software Meshroom, which, with some tweaking of the default parameters, allowed the creation of highly detailed 3D models (figs. 5 and 6). The models are then optimized in Blender and Meshlab. An example of the results is available for download following the link in the caption of fig. 6.

Fig. 5: A phase of the photogrammetric reconstruction of the book’s 3D geometry in the open source software Meshroom.
Fig. 6: The 3D model resulting from the SfM processing (available for interactive viewing and download at this link). This book is the Antiquitatum Romanarum Corpus Absolutissimum by the German antiquarian Johannes Rosinus (1551 – 1626), published in Leiden in 1663 by the Officina Hackiana (private collection). De Graeff owned the 1646 edition in the same quarto format.

Future outlook

Unfortunately the ‘Museum of Bags and Purses’ has been closed as a result of the corona crisis, which has halted our plans to make available these 3D reconstructions at their original location in the future. These digital assets will however remain as geospatial interfaces to visualize the reconstruction hypotheses of this domestic space, and will give access to the structured web of data underlying their creation. Ultimately, in fact, all these sources and the datasets will be linked to the 3D reconstruction of this room via the web-based viewer that Hugo Huurdeman is designing in the context of this project. The viewer will allow the user to experience the 3D reconstruction in both desktop and VR-mode, and is aimed at intellectual transparency by providing various options to access additional information about the reconstruction process: from a color-coded overlay displaying the degree of uncertainty in the reconstructed 3D environment, to the possibility to explore both its underlying sources and additional linked datasets.

Works cited

P. Dijstelberge and P.J. Verkruijsse 2010. ‘Een schitterend moeras: boek en wereld in de zeventiende eeuw’. In Jaarboek voor Nederlandse Boekgeschiedenis 17, 141-170.

P.G. Hoftijzer 2015. ‘The Dutch Republic, Centre of the European Book Trade in the 17th Century’, consulted at http://ieg-ego.eu/en/threads/backgrounds/the-book-market/paul-g-hoftijzer-the-dutch-republic-centre-of-the-european-book-trade-in-the-17th-century

A. Pettegree and A. der Weduwen 2019. The Bookshop of the World. Making and Trading Books in the Dutch Golden Age. Yale University Press.

C. Piccoli (forthcoming). ‘Home-making in 17th century Amsterdam: A 3D reconstruction to investigate visual cues in the entrance hall of Pieter de Graeff (1638-1707)’, in G. Landeschi and E. Betts (eds.), Capturing the senses: digital methods for sensory archaeologies, Springer.

I. H. van Eeghen 1978. De Amsterdamse boekhandel 1680-1725. Deel 5. De boekhandel van de Republiek 1572-1795, Amsterdam.


[1] I would like to thank Paul Hoftijzer, curator of the Thysiana and professor of Book History in Leiden, for the time he made available to discuss both the content and the appearance of De Graeff’s library and for his useful observations and feedback. Thanks also to Gabri van Tussenbroek for checking the structural elements of the reconstructed room.

[2] Alessandro contributed also to the 3D modelling of some objects in the room, specifically a first version of the window, one reconstruction hypothesis of the shelves, and the closet. His thesis is supervised by Maarten Lamers (LIACS) and myself.

The Importance of Context in Search User Interfaces for 3D Content

How to design user interfaces for searching and retrieving 3D models in a digital humanities context? This was the question investigated in our short paper entitled “More than just a Picture” – The Importance of Context in Search User Interfaces for Three-Dimensional Content [1] (PDF), published in the proceedings of the CHIIR conference 2020. Since the conference couldn’t physically take place due to the COVID-19 situation, we briefly present our work in this blogpost.

Background

This paper was inspired by the growing number of historical environments which are reconstructed in three dimensions, for instance the 17th century houses in Amsterdam within the Virtual Interiors project. However, search and retrieval of three-dimensional assets are so far underrepresented in research and practice. Thus, it is unclear how to support complex searches for these types of content, for instance in the context of digital humanities (DH) research.

To investigate interfaces for searching and retrieving 3D models, we looked at three questions: 

1. How do digital humanists conceptualize search in the context of historical reconstructions? 

2. Which spatial support by 2D, enhanced 2D and 3D maps do they prefer for exploratory searches?

3. Which dimensions of spatial context do digital humanists refer to when searching for 3D objects?

The questions were researched via an exploratory user study with six Digital Humanities researchers, using a so-called “think-aloud” protocol, in which participants verbalize their thoughts and actions. We evaluated interactive mockups of a prospective search engine, contextualized by 2D, enhanced 2D and 3D maps.

Figure 1: Basic (‘baseline’) search user interface (first 2 results). Result item: (A) 3D model thumbnail. From archival info.: (B) Title; (C) – (D) descriptions; (E) house address, room; (F) – (H) various metadata. (I) – (J) Degree of uncertainty in proposed 3D reconstruction.

Findings

Our findings are fully described in the paper referenced below [1], but here we summarize the main points:

Conceptualizing a 3D search engine (1)

During our user study, DH researchers imagined how a search engine for 3D objects would work, and what it would look like. They emphasized textual aspects, such as “a list of paintings that were in the house”, item descriptions, links to sources and various enrichments. Also, visual aspects played a key role. For instance, the 3D search engine should be “visually attractive”, one “should know what things looked like” and “not just get the image”, but also the overall spatial and historical context was emphasized. This means not just knowing “where something is”, but also how an object “relates to other objects and environments”, according to the participants.

Comparing 2D & 3D map variants (2)

In a subsequent part of our study, we looked further into which spatial support should ideally be provided in a 3D search engine. Participants tried out interactive mockups containing static 2D, and interactive 2D and 3D maps of a prospective search engine in a randomized order (see Figure 1 and 2). The responses of post-task surveys can best be summarized by participants’ satisfaction ratings: this was clearly highest for the 3D maps (4.2 out of 5) as opposed to enhanced 2D (3.8) and static 2D maps (3). As one participant indicated, the 3D maps “give a better perception of space”. At the same time, a need for combining map modalities came up: the usefulness of the 2D and 3D maps depends on the type of task a researcher is conducting.

Dimensions of spatial context (3)     

Our third question focused specifically at the role of spatial context, via a thematic analysis of the transcribed think-aloud sessions, which derives the main themes and subthemes that occurred for the combined set of participants. The main identified themes were:

  • Object properties ❍, referring to location and size of objects
  • Object relationships , referring to relations between objects with other objects, or the environment 
  • Perception of space , including proportions, structural elements, space functions and lighting

These categories are visualized in Figure 3, which depicts the subcategories of the three main themes, and how many study participants referred to them. The findings represented in this diagram can help to inform design of future search interfaces for 3D objects, and to decide which kind of spatial context is needed.

Conclusion

Summarizing, our exploratory study showed the prime role of context in retrieving historical 3D objects, such as object properties, object relationships and cues in the environment. Participants expressed a preference for 3D contextual maps when presented with different types of maps in a prospective search interface. Our findings suggest a need for hybrid visualization modalities depending on the task at hand, which can be informed by the identified spatial context dimensions. Please refer to our published paper for full details and references.

[1] Hugo Huurdeman & Chiara Piccoli (2020). “More than just a Picture” – The Importance of Context in Search User Interfaces for Three-Dimensional Content. In Proceedings CHIIR 2020. ACM. (PDF).

Visualizing domestic interiors in 17th century Amsterdam: 3D/4D data integration and hypothesis testing platforms

This project in a nutshell:

My postdoc project investigates how individuals created, used, displayed and experienced cultural goods in their homes during the Dutch Golden Age by developing virtual reconstructions of a selection of domestic interiors. These three-dimensional (3D) models aim to act as data integration and hypothesis visualizing platforms to spatially connect, manage and research the rich and heterogeneous data sources on this period. This project aims also to deal with the modelling process in a transparent way and, when possible, to visualize changes through time (4D).

Powered by WordPress & Theme by Anders Norén