Author: Hugo Huurdeman

The Importance of Context in Search User Interfaces for 3D Content

How to design user interfaces for searching and retrieving 3D models in a digital humanities context? This was the question investigated in our short paper entitled “More than just a Picture” – The Importance of Context in Search User Interfaces for Three-Dimensional Content [1] (PDF), published in the proceedings of the CHIIR conference 2020. Since the conference couldn’t physically take place due to the COVID-19 situation, we briefly present our work in this blogpost.

Background

This paper was inspired by the growing number of historical environments which are reconstructed in three dimensions, for instance the 17th century houses in Amsterdam within the Virtual Interiors project. However, search and retrieval of three-dimensional assets are so far underrepresented in research and practice. Thus, it is unclear how to support complex searches for these types of content, for instance in the context of digital humanities (DH) research.

To investigate interfaces for searching and retrieving 3D models, we looked at three questions: 

1. How do digital humanists conceptualize search in the context of historical reconstructions? 

2. Which spatial support by 2D, enhanced 2D and 3D maps do they prefer for exploratory searches?

3. Which dimensions of spatial context do digital humanists refer to when searching for 3D objects?

The questions were researched via an exploratory user study with six Digital Humanities researchers, using a so-called “think-aloud” protocol, in which participants verbalize their thoughts and actions. We evaluated interactive mockups of a prospective search engine, contextualized by 2D, enhanced 2D and 3D maps.

Figure 1: Basic (‘baseline’) search user interface (first 2 results). Result item: (A) 3D model thumbnail. From archival info.: (B) Title; (C) – (D) descriptions; (E) house address, room; (F) – (H) various metadata. (I) – (J) Degree of uncertainty in proposed 3D reconstruction.

Findings

Our findings are fully described in the paper referenced below [1], but here we summarize the main points:

Conceptualizing a 3D search engine (1)

During our user study, DH researchers imagined how a search engine for 3D objects would work, and what it would look like. They emphasized textual aspects, such as “a list of paintings that were in the house”, item descriptions, links to sources and various enrichments. Also, visual aspects played a key role. For instance, the 3D search engine should be “visually attractive”, one “should know what things looked like” and “not just get the image”, but also the overall spatial and historical context was emphasized. This means not just knowing “where something is”, but also how an object “relates to other objects and environments”, according to the participants.

Comparing 2D & 3D map variants (2)

In a subsequent part of our study, we looked further into which spatial support should ideally be provided in a 3D search engine. Participants tried out interactive mockups containing static 2D, and interactive 2D and 3D maps of a prospective search engine in a randomized order (see Figure 1 and 2). The responses of post-task surveys can best be summarized by participants’ satisfaction ratings: this was clearly highest for the 3D maps (4.2 out of 5) as opposed to enhanced 2D (3.8) and static 2D maps (3). As one participant indicated, the 3D maps “give a better perception of space”. At the same time, a need for combining map modalities came up: the usefulness of the 2D and 3D maps depends on the type of task a researcher is conducting.

Dimensions of spatial context (3)     

Our third question focused specifically at the role of spatial context, via a thematic analysis of the transcribed think-aloud sessions, which derives the main themes and subthemes that occurred for the combined set of participants. The main identified themes were:

  • Object properties ❍, referring to location and size of objects
  • Object relationships , referring to relations between objects with other objects, or the environment 
  • Perception of space , including proportions, structural elements, space functions and lighting

These categories are visualized in Figure 3, which depicts the subcategories of the three main themes, and how many study participants referred to them. The findings represented in this diagram can help to inform design of future search interfaces for 3D objects, and to decide which kind of spatial context is needed.

Conclusion

Summarizing, our exploratory study showed the prime role of context in retrieving historical 3D objects, such as object properties, object relationships and cues in the environment. Participants expressed a preference for 3D contextual maps when presented with different types of maps in a prospective search interface. Our findings suggest a need for hybrid visualization modalities depending on the task at hand, which can be informed by the identified spatial context dimensions. Please refer to our published paper for full details and references.

[1] Hugo Huurdeman & Chiara Piccoli (2020). “More than just a Picture” – The Importance of Context in Search User Interfaces for Three-Dimensional Content. In Proceedings CHIIR 2020. ACM. (PDF).

Geographies & enhanced publications at ECARTICO data sprint

On Thursday 12 March, a data sprint regarding the datasets ECARTICO and ONSTAGE took place at the University of Amsterdam, organized by Amsterdam Time Machine / Golden Agents. Since the used datasets are very relevant for the Virtual Interiors project, several team members joined the sprint. 

As indicated on the data sprint website, “ECARTICO contains biographical data about painters, engravers, publishers, goldsmiths, writers and other creative people working in the Low Countries” in the 16th and 17th centuries, and provides information about their networks. Additionally, ONSTAGE contains the complete performance history in the Amsterdamse Schouwburg from 1638 to 1940. Since these datasets are very extensive (ECARTICO alone contains more than 50,000 entries), and since they also provide extensive links to e.g. Wikidata, DBNL and RKD, it was very worthwhile to experiment with these datasets in the sprint.

During the event, team members of Virtual Interiors joined a workshop focused on Geographies and networks of style and subject matter (Workshop 3), and on Enhanced publications (Workshop 4). Workshop 3 looked at the opportunities to use and combine ECARTICO, Wikidata and RKD data to create geographies of style, and social networks. Using “semantic queries” in SPARQL, we could retrieve painters from certain periods, and get statistics on the cities they worked from. For a full overview and explanation of methods, please refer to the page for workshop 3. Workshop 4 focused on enhanced publications, i.e. single-screen applications which allow for simultaneously accessing biography texts, artwork images and other updated biographical information. Also in this case, the full information on project objectives, approaches and results can be found on the dedicated page for workshop 4.

For Virtual Interiors, the results of the data sprint are particularly interesting since they provide inspiration for our integration of linked data, the potential opportunities and difficulties, and because they can inform our explorations into creating enhanced publications.

Powered by WordPress & Theme by Anders Norén