Engelhardt, Y. (2007). Syntactic structures in graphics. Computational Visualistics and Picture Morphology, 5, 23-35.

Engelhardt, Y. (2007). Syntactic structures in graphics. Computational Visualistics and Picture Morphology, 5, 23-35.

圖形的目的是用來說明某種的資訊，使得原本無法看見的訊息能被看見 (visualizing the nonvisual)。依據過去的文獻，本研究建議將圖形的建構(building blocks)分為三個部分： a) 顯示的圖形物件 (graphic objects)、b) 配置這些物件使其具有意義來說明訊息的圖形空間 (graphic spaces)以及c) 這些物件的圖形性質 (graphic properties)。在此，圖形物件為一個遞迴的結構 (recursive structure)，也就是：在一個圖形空間上配置的一組圖形物件能夠共同形成一個更高階的圖形物件。

本研究也提出圖形物件的語法類型 (syntactic categories)，用來解釋物件彼此間可以允許的空間關係並且區分圖形的基本構成 (basic constituents)。所有的圖形都是建立在不同語法類型的圖形物件之組合的可能性上，不同語法類型的圖形物件在圖形呈現 (graphic representation)上有不同的行為，此一限制產生它們有不同的空間定位。所有圖形物件的語法類型可分為兩大群組：1) 依附於圖形空間位置上的物件；2) 依附於其他物件上的物件。前者包括節點 (node)、線標示 (line locator)、面標示 (surface locator)以及格標記 (grid marker)；後者則有標籤 (label)、連結線 (connector)、比例區段 (proportional segment)和框架 (frame)。以地圖為例，節點 、線標示、面標示可分別代表地圖上城市、河流與湖泊或國家的標示，格標記則用於表示地圖上的經緯度，標籤則是標明節點代表的城市名稱。各種圖形物件的類型、依附類型與例子，如下表：




根據語言的語法學概念，圖形的語法學 (the syntactics of graphics)研究不同語法類型的圖形物件之間的關係，研究圖形物件與圖形空間之間以規則與限制為基礎的關係，也研究圖形物件如何組合成複合的圖形物件以及複合的圖形物件如何能以更簡單的圖形物件分析。圖形可分為實體場景和物件的影像以及抽象的圖形，前者如照片與地圖等呈現實體的空間，後者則如家族樹 (family trees)、統計圖表 (statistical charts) 等呈現概念性的空間。照片與地圖等利用影像中的空間配置呈現出真實的空間配置；家族樹和圓餅圖 (pie charts) 則以影像中的空間配置呈現非空間性的資訊。然而，實體空間的呈現並不一定表達出被呈現物件的真實座標比例(the true coordinate proportions)，許多圖形也同時由實體和概念性的空間組合而成，例如在地圖上以高度呈現國家的人口密度分布。下表是若干圖形空間的類型與代表。

Building upon the existing literature, we are suggesting to regard the building blocks of all graphics as falling into three main categories: a) the graphic objects that are shown (e.g., a dot, a pictogram, an arrow), b) the meaningful graphic spaces into which these objects are arranged (e.g., a geographic coordinate system, a timeline), and c) the graphic properties of these objects (e.g., their colors, their sizes).

We suggest that graphic objects come in different syntactic categories, such as nodes, labels, frames, links, etc. Such syntactic categories of graphic objects can explain the permissible spatial relationships between objects in a graphic representation.

In addition, syntactic categories provide a criterion for distinguishing meaningful basic constituents of graphics.

It is about images that can be regarded as ‘visualizing the nonvisual’ in an attempt to clarify information of some sort. Such images are often collectively referred to as “graphics”.

In 1914, Willard Brinton writes in his book Graphic methods for presenting facts that “The principles for a grammar of graphic presentation are so simple that a remarkably small number of rules would be sufficient to give a universal language”.

In 1967, Jacques Bertin publishes his classic Sémiologie graphique, in which he analyses the “language” of graphic representations and the “visual variables of the image”.

In 1976, linguist Ann Harleman Stewart examines the properties of diagrams and claims that “Like any language, graphic representation has a vocabulary and a grammar”.

In 1984, Clive Richards proposes a “grammatically-based analysis” of diagrams in his Ph.D. thesis Diagrammatics.

In 1986, Jock Mackinlay suggests that “graphical presentations are actually sentences of graphical languages that have precise syntactic and semantic definitions”. In Mackinlay’s approach, “the syntax of a graphical language is defined to be a set of well-formed graphical sentences”.

In 1987, Fred Lakin publishes his paper “Visual grammars for visual languages”, in which he describes his approach to the “spatial parsing” of graphics, which he defines as “the process of recovering the underlying syntactic structure of a visual communication object from its spatial arrangement”.

Kress and van Leeuwen publish their book Reading images: the grammar of visual design (1996). Unfortunately, it is difficult to extract a systematic approach to a syntactic analysis of graphics from their book.

A paper titled “The visual grammar of information graphics” (1996) by Engelhardt et al., suggests “syntactic categories of visual components”.

Robert Horn, in his book Visual Language (1998), proposes a morphology and a syntax of visual language based partly on the work of Jacques Bertin and on the Gestalt principles of perception.

In his book The grammar of graphics (1999), Leland Wilkinson describes an approach to graphics that is related to object-oriented design in computer science. However, he uses grammatical terminology “metaphorically”, and not in a linguistic sense.

Colin Ware (2000) writes about the “perceptual syntax of diagrams”, describing “the grammar of node-link diagrams” and “the grammar of maps”.

Engelhardt, in his Ph.D. thesis The language of graphics (2002) provides a detailed proposal for the analysis of syntactic structure, which he applies to a broad spectrum of graphic representations.

We propose a notion of graphic objects that will allow for recursive structures: Any graphic representation – and any meaningful visible component of a graphic representation – may be referred to as a graphic object. This means that graphic objects can be distinguished at various levels of a graphic representation. For example, a map or a chart in its entirety is a graphic object. In addition, the various symbols or components that are positioned within that map or chart are graphic objects as well.

A bottom-up description of this principle was given above: a set of graphic objects can be arranged into a graphic space, together forming a single graphic object at a higher level. This “nesting” or “embedding” (Engelhardt 2002) of graphic structures can be referred to as “recursive composition” (Card 2003).

In technical terms, a meaningful graphic space could be defined as a graphic space that involves an interpretation function from spatial positions to one or more domains of information values.

In graphics, not only the possible constituents themselves (graphic objects), and the diverse possible ways of arranging these constituents (in meaningful graphic spaces), but also the possible visual appearances of these constituents (graphic properties such as size, color), could be considered as being part of the graphic “vocabulary”. In this sense we can say that the building blocks of graphics fall into three main categories: graphic objects, meaningful graphic spaces, and graphic properties.



To make a more general statement, we claim that all graphics are based on the possibility of combining graphic constituents (graphic objects) of different syntactic categories (Engelhardt et al. 1996, Engelhardt 2002, 2006).

Graphic objects of different syntactic categories “behave” differently in a graphic representation. The constraints that govern their spatial positioning are different.




All syntactic categories of graphic objects can be divided into two main groups: 1) objects that are attached to locations in graphic space (e.g., node, line locator, surface locator, grid marker are all attached to locations in graphic space), and 2) objects that are attached to other objects (label, connector, proportional segment, frame are all attached to other objects).

Richards (1984) believes that “there seems to be little profit in using such items as an individual dot or line as a unit of analysis. If we are going to use linguistics as a model, then what is needed for present purposes is not the pictorial equivalent of a phoneme or morpheme but something closer to a noun phrase”.

The basic graphic objects in a particular graphic representation are those that can be regarded as functioning in some syntactic category within that particular graphic representation (e.g., as a label, as a node, as a connector, as a proportional segment, etc.).

The distinction between syntactics, semantics, and pragmatics was introduced by Charles Morris (1938, 1946). Morris conceives of syntactics as the investigation of the relationships between signs, of the ways in which complex signs can be constructed from simple ones, as well as the ways in which complex signs can be analyzed into more simple ones (Morris 1946/1971).

The syntactics of graphics investigates the relationships between graphic objects of different syntactic categories. It investigates the rule- and constraint-based relationships between graphic objects (of different syntactic categories) and graphic spaces.

And syntactics investigates how graphic objects can be combined into composite graphic objects, and how composite graphic objects can be analyzed into more simple ones.

Looking at the broad spectrum of graphics we can say that images of physical scenes and objects, such as pictures and maps, represent physical spaces, while many abstract graphics, such as family trees and statistical charts, represent conceptual spaces (Engelhardt 1999, 2002).

In other words, pictures and maps use spatial arrangement in the image to represent spatial arrangement in the world, while family trees and pie charts use spatial arrangement in the image to represent non-spatial information.

Representations of physical spaces do, by the way, not always have to express the true co-ordinate proportions of the represented objects.

Many graphics combine physical and conceptual spaces.

As an example of a true hybrid space (Engelhardt 1999, 2002), think of a three-dimensional landscape drawing of a country in which the drawn “mountains” do not represent physical mountains, but – for example - population density, peaking in the cities and flat in the countryside. In this case, the horizontal plane represents the physical space of the country’s geography, while the vertical dimension represents the conceptual space of population density.




We claim that all types of graphic representation of information can be analyzed in terms of their composition from graphic spaces of different sorts.

We have tried to show that specifying such a visual language means a) specifying the syntactic categories of its graphic objects, plus b) specifying the graphic space in which these graphic objects are positioned, plus c) specifying the visual coding rules that determine the graphic properties of these graphic objects (see table 1).

The syntactic structure of a graphic representation is determined by the rules of attachment for each of the involved syntactic categories (see table 2) and by the structure of the meaningful graphic space that is involved (see table 3).

With this analysis we have attempted to demonstrate that Morris’ original notion of syntactics applies well to the structure of graphics.

Wanner, F., Stoffel, A., Jäckle, D., Kwon, B. C., Weiler, A., Keim, D. A., ... & Pfister, H. (2014). State-of-the-art report of visual analysis for event detection in text data streams. In Computer Graphics Forum (Vol. 33, No. 3).

Wanner, F., Stoffel, A., Jäckle, D., Kwon, B. C., Weiler, A., Keim, D. A., ... & Pfister, H. (2014). State-of-the-art report of visual analysis for event detection in text data streams. In Computer Graphics Forum (Vol. 33, No. 3).

近年來從文本串流中偵測事件已成為熱門的研究領域，然而由於對事件的概念沒有妥善的定義以及文本資料的種類繁多，對資料分析與視覺化是一個重大的挑戰，因此能夠處理特定事件類型與多樣性文字來源的視覺分析工具的建立準則十分缺乏。在本研究中，將事件視為是從文本資料中抽取出的對使用者有價值的非預期而獨特的樣式(unexpected and unique patterns)，並且建議由新聞標準(news criteria)或新聞價值(news values)[GR65]來界定事件的價值。

在從文本資料串流利用視覺分析進行事件偵測的研究中，資料的來源從有限而書寫良好的新聞文章到社交媒體上由使用者書寫、快速產生甚至有時沒結構的文字資料；而分析任務則可依其目的分為新事件偵測(new event detection)、事件追蹤(event tracking)、事件摘要(event summarization)以及事件關聯(event associations) (Dou et al., DWRZ12)。Becker [Bec11]對於社交媒體上的事件偵測研究，將事件依據3個面向區分：1) 計畫內 (planned) vs. 無計畫 (unplanned)、 2) 趨勢 (trending) vs 非趨勢 (non-trending)、 3) 外源 (exogenous) vs. 內源 (endogenous)。

本研究以Figure 1上的流程圖表示事件偵測與探索的處理過程，首先在輸入文件資料的前處理 。在前處理之後的方法，則可分為兩類：一類首先應用自動化方法偵測資料裡的事件，然後再利用這些資訊做為視覺分析(visual analysis)的介面；另一類則是直接對前處理後的結果進行視覺化，不進行事件的自動化分析。






以下分別說明文本資料來源、文本處理方法等技術分析的面向。

文字資料來源包括：新聞、電子郵件、部落格、RSS feed、微網誌 (microblogging) 訊息、論壇(forum)上的發文(post)、客服表單、影像與視頻串流上附註的文字。目前有大半的研究是針對微網誌資料，例如Twitter，而除了文字以外，微網誌上的地理位置和作者等後設資料也是許多研究會加以利用的。

在文句偵測(sentence detection)、(tokenizing)、詞幹化(stemming)和(lemmatizing)等文本處理方法之後，進行較深入的詞類標示(part-of-speech tagging)、語法剖析(syntactic parsing)、文句中詞語關係的類型剖析 (Typed-dependency parsing)、相互指涉解析(coreference resolution)、專有名詞辦認 (named entity recognition)、極性抽取 (polarity extraction)、歧義消除 (word sense disambiguation)等。自2000到2011年，33篇事件偵測的相關論文只有17篇利用文本處理方法，但在2012年後，這個情形改變了，18篇論文中便有14篇論文使用文本處理方法，主要是詞類標示和極性抽取。

事件偵測的自動化方法中常用的技術可分為1)群集為基礎、2)以分類為基礎、3)以統計為基礎、4)以預測為基礎、5)本體論為基礎、6)模式探勘(pattern mining)、7)資料串流重複特徵的模型、8)規則式等類型。
以群集為基礎的方法將文件依據內容的不同特性分群，當群組改變時便是事件產生。
分類則是根據事件建立分類器，當文件的分類結果為事件相關，便視為是事件產生。
統計方法中，相關分析類的方法測量文件集合在詞語或詞語與時間上的相關性改變來偵測事件，另一類則是從稀少或獨特的詞語出現來發現事件。
預測為基礎的方法根據過去的歷史預測接下來文件的出現情形。
本體論(ontologies)為基礎的方法適合單一領域的事件偵測，以全自動或半自動方法產生特定本體，當偵測到活躍概念(activated concepts)中的改變時便是可能的事件。

模式探勘(pattern mining)利用A-priori 演算法抽取文件串流上的常見連續模式。
資料串流重複特徵的模型可以用來偵測事件，當一個串流明顯偏離它預期的特徵時便是偵測到一個事件。
規則為基礎的方法以人工編寫的規則偵測事件，例如根據詞語或詞頻為規則來偵測特定的事件。

在視覺化呈現上，以時間為基礎的視覺化呈現佔有大多數，共21篇論文，利用包括河流 (river)、時間線 (timeline)與圓形 (circular)等時間為基礎的呈現凸顯資料在時間上的演變。時間線的呈現會將符號放置在一或多條時間線上，表現資料項目、密度與數量，能夠表現單一或稀少的事件。河流通常用來做為群集演算法結果的視覺化，提供各群集的分布與整體的數量，較著重在高頻率的事件上。以微網誌為資料來源的應用，通常會利用微網誌上附加的地理參考資料，以地圖的方式來呈現。折線圖或長條圖等基本的視覺化呈現方式通常運用來表現事件有關的資料在時間上的數量與頻率。在各種視覺化的應用中，文字資料的呈現通常選用具有意義的關鍵詞。

在支援的分析任務上，包括 1) 提供文件集合的概觀，描述集合內發現的主題，利於進一步的分析。2) 關鍵詞語搜尋以及利用後設資訊過濾資料，在分析或視覺化時減少資料或抽取的事件數量。3) 監測資料來源中事件在時間上的發展。4) 將偵測到的不同事件間的關係視覺化。

質性的評估方法包括案例研究(case study)、使用性評估(usability evaluation)、使用案例(use case)以及軼事評估(anecdotal evaluation)，其中使用案例最為盛行。較常見的量化評估方式比較偵測到的事件與真實的資料。

Event detection from text data streams has been a popular research area in the past decade.

However, data analysts and visualization experts often face grand challenges stemming out of the ill-defined concept of event and various kinds of textual data. As a result, we have few guidelines on how to build successful visual analysis tools that can handle specific event types and diverse textual data sources.

Within this paper, events are regarded as unexpected and unique patterns extracted from text data streams, valuable to users.

In particular, data sources evolved from a relatively limited amount of well-written news articles to rapidly generated, user written, and in some cases unstructured textual data from social media services.

Dou et al. [DWRZ12] defined task according to “New Event Detection”, “Event Tracking”, “Event Summarization”, and “Event Associations”, but we expect that tasks can be even more diversified including geographic dimension which were not explored yet.

Another challenge represents the unstructured, diverse textual data. It mandates extensive processing and preparation in order to properly employ it.

Becker [Bec11] shows interesting work about event detection in social media. She divides an event using three dimensions: 1) “planned” vs. “unplanned”; 2) “trending” vs “non-trending”; 3) “exogenous” vs. “endogenous”. The last dimension aims to detect events within the data in a real-life context.

Some examples of visual social media analysis is shown in Schreck and Keim [SK13]. With screenshots of the different visualizations, the authors explain the underlying data, analysis methods, and functionality of various applications in visual social media analysis.

There exists a survey on semantic sensemaking by Bontcheva and Rout [BR12]. Though their focus was on the semantic aspects, a subsection refers to visualization approaches.

Rohrdantz et al. [ROKF11] mention tasks for the “RealTime Visualization of Streaming Text Data”. They call tasks that are relevant in terms of the scope of our paper “monitoring”, “change and trend detection” and “situational awareness”.

In the first step of the pipeline, the documents are prepared for the analysis. In this step the documents are parsed to get the plain texts and standard text preprocessing methods, such as sentence detection, tokenizing, and stemming and lemmatizing are applied. In addition to these standard methods, methods from the computer linguistic field can be used in the preprocessing step to annotate the texts with additional information. For instance, part-of-speech tagging, named entity extraction, or syntactic parsing can be used to identify types of words, persons and places, or structure of sentences.

After the preprocessing step different approaches are used to detect events (see two branches in Processing in Figure 1).
The first group of approaches applies automatic methods to detect patterns in the data. The detected patterns are then used to create a visual analysis interface for the data set, what we call visual analysis. The interaction between visualization and the automatic part shapes a visual analytics approach.
The second group of approaches skips the automatic analysis and directly visualizes the outcome of the preprocessing, what also is only visual analysis because of the lack of interaction possibilities of a certain extent.

Text Data Sources

1. News is a well-known text data source. News captures information of a real world event or happening. It consists of a title, often followed by a short summary and the body containing details about the event. News goes through a professional gatekeeping process which in the end forms the agenda of media.

2. A typical electronic document is email. ... Emails are used for personal conversations, advertisement or business information exchange. They consist of a header and a body. The header contains information about transaction: sender, receiver, timestamp, and other meta data. The body contains the textual content of the email. An email body can be of arbitrary length which is one of its characteristics.

3. Weblogs, shortly named blogs are used for information purposes of a more or less undefined audience. ... A blog can have a specific topic or can be open for various topics.

4. RSS feeds are a standardized format to broadcast short news snippets. They consist of a title and a description. RSS feeds can be used by news agencies, newspapers and blogs. ... The standardized format allows the easy integration into other applications.

5. Recently, microblogging providers are becoming more and more popular. The messages are limited with respect to their length of 140 characters. So-called “hashtags” are used in order to characterize the membership of a tweet to a certain topic. In addition, more meta data is provided, e.g. geolocation, author, place etc.

6. User forums often have hierarchical structure. A message within the forum is a post and is not strictly restricted with respect to its length. Posts which belong to the same topic shape a so called thread. On the other hand several threads often belong to a sub-forum within the main forum. The purpose of a forum is the discussion on specific issues and topics regarding the its main topic.

7. Modern customer-care systems often ask each customer to fill out a feedback form after a purchase. This form (often digital, reachable through the internet) gives the customer the opportunity to provide issues directly to the vendor. The information is a valuable source which allows the seller to react fast and adequately to issues being raised by customers. ... Often these forms are semi-structured, which means they have checkboxes for predefined questions and provide a free text field for further comments.

8. Images and video sequences can be uploaded on sharing sites such as Flickr (https://www.flickr.com/). Users can tag their content with text. These tags and little text snippets typically describe the content in a short manner or express an emotional state being associated with the photo.

Almost half of the papers use microblogging data namely Twitter. It is obvious in Table 1 that in 2010 a shift towards microblogging happened. It is also noticeable that meta data (geolocations, author information) is often used in conjunction with microblogs.

Text Processing Methods

1. Part-of-speech (POS) tagging detects the word type of tokens.

2. Syntactic parsing determines the grammatical structure of sentences. ... Full syntactic parsing uses grammars and build up a complete parse tree for a sentence. ... Shallow parsing creates meaningful chunks and avoids the complexity of full parsing.

3. Typed-dependency parsing determines the type of relations between words in a sentence.

4. Coreference resolution creates connection between referring expression, such as pronouns, and subjects in a text. A correct resolution of referring expressions could improve text mining results, e.g., polarity extraction would benefit from correctly resolved referring expressions.

5. Named entity recognition (NER) detects and labels names of, e.g., persons, locations, events, or dates in texts.

6. Polarity extraction or determines the attitude (positive vs. negative) of the writer about a subject.

7. Word-sense disambiguation techniques use the context of words to determine the correct sense of tokens. ... We only observed one paper using word sense disambiguation.

We confirm that text processing methods are used very sparingly. ... Since 2000 until the end of 2011, only 17 out of 33 papers utilized any of the methods. The 16 papers with no text processing methods solved the event detection tasks with visualization. In the year of 2012, the trend changed dramatically; 14 out of 18 papers have used text processing methods in the papers published since then.

It is also noticeable that part-of-speech tagging and polarity extraction have gained popularity since 2012 as well.

Thus, we believe that many research papers started absorbing more natural language processing techniques to further generate their event metrics.

Automatic Methods for Text Event Detection

1. Clusters are generated for different time windows based different properties in the document, e.g., co-occurrence of terms, frequency in time, or metadata. Events are generated when the set of clusters changes, e.g., a new cluster arise or two existing clusters merge.

2. Users provide a set of example documents and classifiers learn to detect the annotated events. Classifier-based techniques are used in similar cases with rule-based ones, but have the advantage that users do not need to create rules by themselves.

3. Statistical methods such as correlation or detection of outliers and significant difference are used to identify events. Correlation based methods examine collection between terms or between terms and time and detect events by changes in the correlation measures. A different type of statistical methods calculate term-wise deviation from an expected value or use other measures to identify rare or unique occurrence of terms.

4. Prediction-based methods predict the occurrence of following documents based upon past history.

5. Methods based on ontologies [HHSW09] are suitable for event analysis in single domains. Specific ontologies are generated with full- or semi-automatic methods. ... Using this type of methods, events can then be detected from changes in activated concepts.

6. Pattern mining algorithms, such as the A-priori algorithm of Wu and Chen [WC09] applied to text in [WSJ∗ 14], are used to extract common sequential patterns in document streams. Patterns can be found based on documents themselves or time intervals. In both cases, features extracted from documents are then used to define patterns.

7. Models of the recurring characteristics of data steams can be used to detect events. An event is detected when a stream deviates significantly from its expected characteristics.

8. Rule-based approaches detect events with manually created rules. For instance, users specify rules based on terms and/or frequency to detect a particular event.

Visualization of Events in Text Data

In total, 21 papers use a time-oriented visualization (river, timeline, circular) to visualize the evolution of the data over time. Time-oriented visualizations are often combined with additional visualizations to show non-time dependent information.

Maps visualization came up with microblogging data and use mainly geographic references in the meta information of the microblogs for visualization.

A problem for all visualizations is the question how to visually represent text data. This problem is usually solved by selecting meaningful keywords that are either generated by frequency or by another scoring technique such as topic models.

Basic visualizations (e.g. line or bar charts) are mainly used to give an overview of the data set by showing the time dependent relations of events. They are used to visualize the data volumes or frequencies over time, for instance, of detected topics, named entities, or keywords.

Timeline visualization use one or multiple timelines and place glyphs or shapes on these timelines to indicate single data items, densities, or volumes. Timeline visualizations are therefore preferred over river visualizations when single or rare items should be tracked, because a river visualization put the focus on high frequent events.

River metaphors are often used to visualize outcomes of cluster algorithms. Although timeline techniques could be used, rivers provide a space saving overview and give a better visual impression of the distributions of the clusters and the overall amount of data.

 Supported Analysis Tasks

1. Overview visualization give users a summary of the document collection. Common are textual summaries based on frequent terms or topic models that describe the topics found in the collection. These summaries serve as navigation support and are often used as starting point for further analysis.

2. It is also common to provide users with abilities to search for keywords or allow filtering of the data by meta information. Both tasks reduce the number of item or extracted events in the analysis or visualization.

3. Monitoring tasks are the second most frequent tasks supported by the surveyed systems. Users monitoring a data source are interested in the evolution of events in a changing data source. Time-based visualizations (e.g., timeline, river, circular) are often used for monitoring task, because they show the temporal development of events in data sources.

4. In many cases relations between different detected events are visualized. The most frequent shown relations are relation in content, time, and volume. For instance, a river visualization shows time and volume relations between different streams and with additional annotations also relations in content can be shown.

The statistical methods are combined with any type of visualizations.

Interestingly, clustering methods are often visualized by river visualizations.

Exceptionally, topic modeling techniques are not only used with time dependent visualizations but also with other visualizations such as treemaps or geographic visualizations. This pattern appears because topic models are clustering methods that return a ranked list of terms representing single topics, which are often used in visualizations to label data and find names for clusters.

Evaluation

We subdivide qualitative methods into the following categories: case study, usability evaluation, use case, and anecdotal evaluation.

Table 7 accentuates the popular usage of use cases; except for 16 of all considered papers the authors make use of this method. Typically, a use case validates through the description of a fictitious scenario that pinpoints main features whereas a case study involves a domain expert and therefore is more time-consuming [DNKS10,MBB∗ 11].

Anecdotal evaluation describes how the suggested system could be used, but do not provide sufficient evidence to judge the general efficacy of the presented technique.

Usability evaluations involve users performing particular tasks with the given system and asks for comments on usability.

The most prominent quantitative evaluation methods are comparisons of the detected events with a ground truth set. Often event databases are used as ground truth that are enriched by the authors with missing entries.

A different evaluation form of algorithms are comparison with existing algorithms and reporting quality measures. In some cases not the results of the algorithms are evaluated but the performance in the sense of runtime or memory consumption is assessed, which is important for systems working in near real-time scenarios.

We also found only four papers using a user study for evaluation. We expected more papers using user studies, because many systems present novel visualization techniques and user studies can verify the strength and weakness of the application [HHN00,LYK∗ 12,RHD∗ 12].

One thing we noticed was that data sources have dramatically changed from news to social media since 2010. Mainly due to the burst of social media, many research studies used text data streams generated out of Facebook or Twitter.

Some data sources – like for instance discussion forums – are underused than others. Discussion forums are traditional methods to collect opinions from many people, but few research topics investigate data because they are asynchronous and slow to build up in nature. Despite these limitations, they also have a strength: archival history of some topics. Several discussion forums include years of textual conversation between multiple users on a single topic. For instance, this longitudinal conversation can be used to detect certain noticeable shifts in a specific user group’s opinions on political issues over some months or years.

More importantly, visualizations were primarily used as presentation, but had no interaction possible to steer the underlying data processing algorithm in order to further analyze data in a different angle. This limitation can prevent users from providing their insights back into the visualizations.

Especially for news, news criteria (also known as news values) [GR65, HO01] can help find and develop new features for content-based feature detection. They are only mentioned once in our whole bulk of surveyed news analysis research papers [DNKS10].

According to [GR65], news criteria are: frequency, threshold, unambiguity, meaningfulness, consonance, unexpectedness, continuity, composition, reference to elite nations, reference to elite people, reference to persons, and reference to something negative.

In general, news and selection criteria could be merged into one concept we call event values. Event values are a concept including the text data producer’s and user’s perspectives. They could be implemented in the data analysis process by means of new features (feature engineering) and interactive elements, which comes along with the call for more visual analytics functionality.

Borkin, M., Vo, A., Bylinskii, Z., Isola, P., Sunkavalli, S., Oliva, A., & Pfister, H. (2013). What makes a visualization memorable?. Visualization and Computer Graphics, IEEE Transactions on, 19(12), 2306-2315.

Borkin, M., Vo, A., Bylinskii, Z., Isola, P., Sunkavalli, S., Oliva, A., & Pfister, H. (2013). What makes a visualization memorable?. Visualization and Computer Graphics, IEEE Transactions on, 19(12), 2306-2315.

過去許多視覺化專家認為視覺化的結果不應該包括「圖表上的廢話」(chart junk)，而應該越清楚地顯示資料越好，例如Edward Tufte 與Stephen Few[13, 14, 37, 38]，心理學的實驗結果也證實了簡單而清楚的視覺化較容易閱讀[11, 24]。但近來也有一些研究者發表「圖表上的廢話」也許能夠增進閱讀者的持久力，讓他們更能努力在了解圖表上，增加對資料的認識與知識[4, 8, 19]。Bateman et al. [4] 的研究指出有裝飾的圖表比素樸的圖表有較佳的記憶，並且在理解上也不會較差，神經生物學則有加上視覺困難 (visual difficulties) 可以增強觀看者理解的假說[8, 19]。除了「圖表上的廢話」之外，圖表的類型、顏色與其他美學因素也會影響人們的觀看、解讀與記憶。本研究利用410張具有圖表的媒體，對261位受試者進行圖表記憶的研究，探討圖表是否與自然景象的影像[20]一樣在不同人之間具有一致性，並且測試什麼樣的視覺化類型與屬性具有較好的可記憶性。

為了進行本研究對各種圖表類型與屬性是否影響圖表的可記憶性，本研究首先參考前人的研究制訂視覺化的分類架構 (taxonomy)。過去的分類架構有根據圖形的知覺模型 (graphical perception models)、視覺與組織配置 (visual and organizational layout) 以及圖形資料的編碼 (graphical data encodings) [6, 12, 35, 33]，或是根據視覺化的演算法 [32, 36] ，近年也有以互動型視覺化和其提供的任務作為分類架構 [16, 17, 33, 35]。本研究的分類架構以Harris [15]提出的詞彙 (vocabulary) 為主，強調圖形表示的語法結構與資訊類型 (Englehardt [12]) 以及人類對圖形的知覺 (Cleveland and McGill [11])，將視覺化分為12類，每一類又分為若干次分類，如下表所示：

在分類架構上，另外對這些圖表附加上若干性質(properties)與屬性(attributes)，性質有維度數目 (2D或3D)、多重性 (單一、群體、多板塊、組合)、插圖(pictorial)以及時間序列，屬性尚包括是否是黑白、顏色的數目、資料與非資料間的比例 (data-ink ratio)、視覺元素面積佔整個影像的比例、是否有插圖以及是否有人像等。

不同的視覺化影像來源中，科學出版品 (scientific publications) 和資訊圖表 (infographics) 在視覺化上具有的多重性非當高，顯示這兩種影像來源的製作上往往需要組合多種圖表來表示概念和理論，而另一個考量則在於節省篇幅；反之，政府與組織出版品則大多是單一視覺化。

以視覺化的類型而言，為了解釋文章中的概念或是理論的結果，科學出版品有極高的比例使用diagram，此外科學文章中也大量使用基本的視覺編碼技術(visual encoding techniques)，諸如折線圖、長條圖和散布圖，某些領域內會使用特定的視覺編碼，例如網格與矩陣圖 (grid and matrix plots)以及樹狀與網路圖 (tree and networks)。因此，相較於其他類型的來源，科學出版品中較'常使用樹狀與網路圖、網格與矩陣圖和散布圖。

資訊圖表中也使用大量的diagrams，主要包括流程圖(flow charts)和時間表(timelines)，此外表格(table)也經常被使用在資訊圖表中，這些表格通常會加上插畫的裝飾。但是資訊圖表相較於其他來源，較少使用折線圖。

新聞媒體和政府出版品主要使用長條圖、折線圖地圖和表格，折線圖通常用來表示時間序列的資料。兩種來源的差異，是政府的報告會大量使用圓形圖，例如圓餅圖。

本研究在評估圖表具有可記憶性的高低時，使用命中率 (hit rate, HR) 與誤警率 (false alarm rate, FAR)，並且用敏感指數(sensitivity index) d-prime metric整合兩個數值進行排序， d' = Z(HR) − Z(FAR) ，Z 是高斯分布(Gaussian distribution)的累積分配函數(cumulative distribution function, CDF)的倒數(inverse)。愈高的d'值表示HR較高而FAR較低，此時該視覺化圖形較容易被記憶，反之則否。本研究得到視覺化圖形的可記憶性平均測量結果為HR = 55.36%和SD = 16.51%以及FAR = 13.17%, SD = 10.73%。相較於先前的圖像可記憶性的研究結果，較自然景象的可記憶性 (HR = 67.5%, SD = 13.6%以及FAR = 10.7%, SD = 7.6%) [21] 差，但與人臉的可記憶性 (HR = 53.6%, SD = 14.3%以及FAR = 14.5%, SD = 9.9%) [2]差不多。而將本研究的測試結果隨機分為兩群，經過25次的測量，HR、FAR和d-prime在兩群間的結量結果的Spearman等級相關係數分別為0.83、0.78和0.81。由此可知，視覺化圖形在可記憶性的不同受試者測量上也有一致性，可記憶性是視覺化圖形的一種特性。

以下針對視覺化圖形的種類、性質與屬性進行可記憶性的探討；

具有插畫的視覺化圖形明顯比沒有插畫的視覺化圖形有較高的可記憶性，

愈多顏色的視覺化圖形的可記憶性愈佳，甚至在沒有插畫的圖形上，較多顏色的圖形也比只有一種顏色的圖形可記憶性來得高。

視覺元素面積佔整個影像的比例較高的視覺化圖形也比較容易被記憶。

資料與非資料間的比例較低的視覺化圖形的可記憶性較佳。

diagram、網格與矩陣圖以及樹狀與網路圖等類型較具有獨特(unique)呈現的圖形比長條圖、折線圖、表格等較一般(common)呈現的圖形較具有可記憶性，特別是圖形上沒有插圖時，因為這類圖形的呈現相近，彼此干擾，使得命中率較低而誤警率較高。

在各種來源的視覺化圖形裡，資訊圖表類具有最高的可記憶性，其次是科學出版品，最不具可記憶性的圖形來源則是政府與世界組織類。

另外，具有圓形或圓的邊的圖形的可記憶性也較高，作者認為插畫與圓形都是較自然的視覺化作品，因此較容易被記憶。


We ran the largest scale visualization study to date using 2,070 single-panel visualizations, categorized with visualization type (e.g., bar chart, line graph, etc.), collected from news media sites, government reports, scientific journals, and infographic sources. Each visualization was annotated with additional attributes, including ratings for data-ink ratios and visual densities.

Altogether our findings suggest that quantifying memorability is a general metric of the utility of information, an essential step towards determining how to design effective visualizations.

The conventional view, promoted by visualization experts such as Edward Tufte and Stephen Few, holds that visualizations should not include chart junk and should show the data as clearly as possible without any distractors [13, 14, 37, 38]. This view has also been supported by psychology lab studies, which show that simple and clear visualizations are easier to understand [11, 24].

At the other end of the spectrum, researchers have published that chart junk can possibly improve retention and force a viewer to expend more cognitive effort to understand the graph, thus increasing their knowledge and understanding of the data [4, 8, 19].

What researchers agree on is that chart junk is not the only factor that influences how a person sees, interprets, and remembers a visualization. Other aspects of the visualization, such as graph type, color, or aesthetics, also influence a visualization’s cognitive workload and retention [8, 19, 39].

Recent work has shown that memorability of images of natural scenes is consistent across people, suggesting that some images are intrinsically more memorable than others, independent of an individual’s contexts and biases [20].

This is because given limited cognitive resources and time to process novel information, capitalizing on memorable displays is an effective strategy.

Recent large-scale visual memory work has shown that existing categorical knowledge supports memorability for item-specific details [9, 22, 23]. In other words, many additional visual details of the image come for free when retrieving memorable items.

For our research, we first built a new broad taxonomy of static visualizations that covers the large variety of visualizations used across social and scientific domains. These visualization types range from area charts, bar charts, line graphs, and maps to diagrams, point plots, and tables.

We then used these 2,070 visualizations in an online memorability study we launched via Amazon’s Mechanical Turk with 261 participants. This study allowed us to gather memorability scores for hundreds of these visualizations, and determine which visualization types and attributes were more memorable.

Perception Theory and the Chart Junk Debate:

Bateman et al. conducted a study to test the comprehension and recall of graphs using an embellished version and a plain version of each graph [4]. They showed that the embellished graphs outperformed the plain graphs with respect to recall, and the embellished versions were no less effective for comprehension than the plain versions.

There has been some support for the comprehension results from a neurobiological standpoint, as it has been hypothesized that adding “visual difficulties” may enhance comprehension by a viewer [8, 19].

Other studies have shown that the effects of stylistic choices and visual metaphors may not have such a significant effect on perception and comprehension [7, 39].

In response to the Bateman study, Stephen Few wrote a comprehensive critique of their methodology [14], most of which also applies to other studies. A number of these studies were conducted with a limited number of participants and target visualizations. Moreover, in some studies the visualization targets were designed by the experimenters, introducing inherent biases and over-simplifications [4, 8, 39].

Visualization Taxonomies:

Within the academic visualization community there have been many approaches to creating visualization taxonomies.

Traditionally many visualization taxonomies have been based on graphical perception models, the visual and organizational layout, as well as the graphical data encodings [6, 12, 35, 33].

Another approach to visualization taxonomies is based on the underlying algorithms of the visualization and not the data itself [32, 36].

There is also recent work on taxonomies for interactive visualizations and the additional tasks they enable [16, 17, 33, 35].

Outside of the academic community there is a thriving interest in visualization collections for the general public. For example, the Periodic Table for Management [26] present a classification of visualizations with a multitude of illustrated diagrams for business. The online community Visualizing.org introduces an eight-category taxonomy to organize the projects hosted on their site [25]. InfoDesignPatterns.com classifies visualization design patterns based upon visual representation and user interaction [5].

Cognitive Psychology:

These studies have demonstrated that the differences in the memorability of different images are consistent across observers, which implies that memorability is an intrinsic property of an image [21, 20].

Brady et al. [9] tested the long-term memory capacity for storing details by detecting repeat object images when shown pairs of objects, one old and one new. They found that participants were accurate in detecting repeats with minimal false alarms, indicating that human visual memory has a higher storage capacity for minute details than was previously thought.

More recently, Isola et al. have annotated natural images with attributes, measured memorability, and performed feature selection, showing that certain features are good indicators of memorability [20, 21]. Memorability was measured by launching a “Memory Game” on Amazon Mechanical Turk, in which participants were presented with a sequence of images and instructed to press a key when they saw a repeat image in the sequence. The results showed that there was consistency across the different participants, and that people and human-scale objects in the images contribute positively to the memorability of scenes. That work also showed that unusual layouts and aesthetic beauty were not overall associated with high memorability across a dataset of everyday photos [20].


The taxonomy classifies static visualizations according to the underlying data structures, the visual encoding of the data, and the perceptual tasks enabled by these encodings.

It contains twelve main visualization categories and several popular sub-types for each category. In addition, we supply a set of properties that aid in the characterization of the visualizations.

This taxonomy draws from the comprehensive vocabulary of information graphics presented in Harris [15], the emphasis on syntactic structure and information type in graphic representation by Englehardt [12], and the results of Cleveland and McGill in understanding human graphical perception [11].

Dimension represents the number of dimensions (i.e., 2D or 3D) of the visual encoding.

Multiplicity defines whether the visualization is stand-alone (single) or somehow grouped with other visualizations (multiple). We distinguish several cases of multiple visualizations. Grouped means multiple overlapping/superimposed visualizations, such as grouped bar charts; multi-panel indicates a graphic that contains multiple related visualizations as part of a single narrative; and combination indicates a graph with two or more superimposed visualization categories (e.g., a line plot over a bar graph).

The pictorial property indicates that the encoding is a pictogram (e.g., a pictorial bar chart). Pictorial unit means that the individual pictograms represent units of data, such as the Istotype (International System of Typographic Picture Education), a form of infographics based on pictograms developed by Otto Neurath at the turn of the 19th century [29].

Finally, time is included, specifically as a time series, as it is such a common feature of visualizations and dictates specific visual encoding aspects regarding data encoding and ordering.

The first two attributes, black & white and number of distinct colors give a general sense of the amount of color in a visualization.

A measure of chart junk and minimalism is encapsulated in Edward Tufte’s data-ink ratio metric [37], which approximates the ratio of data to non-data elements.

The visual density rates the overall density of visual elements in the image without distinguishing between data and non-data elements.

Finally, we have two binary attributes to identify pictograms, photos, or logos: human recognizable objects and human depiction. We explicitly chose to have a separate category for human depictions due to prior research indicating that the presence of a human in a photo has a strong effect on memorability [21].

There is also a very high percentage of multiple visualizations in the scientific publication category. There are two primary explanations for this observation. First, like infographics, multiple individual visualizations are combined in a single figure in order to visually explain scientific concepts or theories to the journal readers. Second, combining visualizations into a single figure (even if possibly not directly related) saves page count and money.

In contrast, a very high ratio of single visualizations is seen in government / world organizations. These visualizations are usually published one-at-a-time within government reports, and there are no page limits or space issues as with scientific journals.

Scientific publications, for example, have a large percentage of diagrams. These diagrams are primarily used to explain concepts from the article, or illustrate the results or theories. Also included are renderings (e.g., 3D molecular diagrams). The scientific articles also use many basic visual encoding techniques, such as line graphs, bar charts, and point plots. Domain-specific uses of certain visual encodings are evident, e.g., grid and matrix plots for biological heat maps, trees and networks for phylogenic trees, etc.

Infographics also use a large percentage of diagrams. These diagrams primarily include flow charts and timelines. Also included in infographics is a large percentage of tables. These are commonly simple tables or ranked lists that are elaborately decorated and annotated with illustrations. Unlike the other categories, there is little use of line graphs.

In contrast to the scientific and infographic sources, the news media and government sources publish a more focused range of topics, thus employing similar visualization strategies. Both sources primarily rely on bar charts and other “common” (i.e., learnt in primary school) forms of visual encodings such as line graphs, maps, and tables. The line graphs are most commonly time series, e.g., of financial data. One of the interesting differences between the categories include the greater use of circle plots (e.g., pie charts) in government reports.

Looking at specific visualization categories, tree and network diagrams only appear in scientific and infographic publications. This is probably due to the fact that the other publication venues do not publish data that is best represented as trees or networks. Similarly, grid and matrix plots are primarily used to encode appropriate data in the scientific context. Interestingly, point plots are also primarily used in scientific publications. This may be due to either the fact that the data being visualized are indeed best visualized as point plot representations, or it could be due to domain-specific visualization conventions, e.g., in statistics.

Worth noting is the absence of text visualizations from almost all publication venues. The only examples of text based visualizations were observed in the news media. Their absence may be explained by the fact that their data, i.e., text, is not relevant to the topics published by most sources. Another possible explanation is that text visualizations are not as “main stream” in any of the visualization sources we examined as compared to other visualization types.

Performance Metrics:

Workers saw each target image at most 2 times (less than twice if they prematurely exited the game). We measure an image’s hit rate (HR) as the proportion of times workers responded on the second (repeat) presentation of the image. In signal detection terms: HR = HITS/(HITS+MISSES).

We also measured how many times workers responded on the first presentation of the image. This corresponds to workers thinking they have seen the image before, even though they have not. This false alarm rate (FAR) is calculated: FAR = FA/(FA+CR) , where FA is the number of false alarms and CR is the number of correct rejections (the absense of a response).

For performing a relative sorting of our data instances we used the d-prime metric (otherwise called the sensitivity index). This is a common metric used in signal detection theory, which takes into account both the signal and noise of a data source, calculated as: d' = Z(HR) − Z(FAR) (where Z is the inverse cumulative Gaussian distribution). A higher d' corresponds to a signal being more readily detected. Thus, we can use this as a memorability score for our visualizations. A high score will require the HR to be high and the FAR to be low. This will ensure that visualizations that are easily confused for others (high FAR) will have a lower memorability score.

In other words, we have measured how visualizations would be remembered if they were images. We observed a mean HR of 55.36% (SD = 16.51%) and mean FAR of 13.17% (SD = 10.73%).

For comparison, scene memorability has a mean HR of 67.5% (SD = 13.6%) with mean FAR of 10.7% (SD = 7.6%) [21], and face memorability has a mean HR of 53.6% (SD = 14.3%) with mean FAR of 14.5%(SD = 9.9%) [2]. This possibly supports our first hypothesis that visualizations are less memorable than natural scenes.

This demonstrates that there is memorability consistency with scenes, faces, and also visualizations, thus memorability is a generic principle with possibly similar generic, abstract, features.

We also measured the consistency of our memorability scores [2, 21]. By splitting the participants into two independent groups, we can measure how well the memorability scores of one group on all the target images compare to the scores of another group (Fig. 3).

Averaging over 25 such random half-splits, we obtain Spearman’s rank correlations of 0.83 for HR, 0.78 for FAR, and 0.81 for d-prime, the latter of which is plotted in Fig. 3.

This high correlation demonstrates that the memorability of a visualization is a consistent measure across participants, and indicates real differences in memorability between visualizations.

In other words, despite the noise introduced by worker variability and by showing different image sequences to different workers, we can nevertheless show that memorability is somehow intrinsic to the visualizations.

Of our 410 target visualizations, 145 contained either photographs, cartoons, or other pictograms of human recognizable objects (from here on out referred to broadly as “pictograms”). Visualizations containing pictograms have on average a higher memorability score (Mean (M)=1.93) than visualizations without pictograms (M = 1.14,t(297) = 13.67, p < 0.001). This supports our second hypothesis.

Thus not all chart junk is created equal: annotations and representations containing pictograms are across the board more memorable.

Thus an image, or image of a visualization, containing a human recognizable object will be easily recognizable and probably memorable. Due to this strong main effect of pictograms, we examined our results for both the cases of visualizations with and without pictograms. As shown in the left-most panel of Fig. 1, all but one of the overall top most memorable images (as ranked by their d-prime scores) contain human recognizable pictograms.

As shown in Fig. 4, there is an observable trend of more colorful visualizations having a higher memorability score: visualizations with 7 or more colors have a higher memorability score (M = 1.71) than visualizations with 2-6 colors (M = 1.48,t(285) = 3.97, p < 0.001), and even more than visualizations with 1 color or black-and-white gradient (M = 1.18,t(220) = 6.38, p < 0.001).

When we remove visualizations with pictograms, the difference between visualizations with 7 or more colors (M = 1.34) and those that have only 1 color (M = 1.00) remains statistically significant (t(71) = 3.61, p < 0.001).

Considering all the visualizations together, we observed a statistically significant effect of visual density on memorability scores with a high visual density rating of “3” (M = 1.83), i.e., very dense, being greater than a low visual density rating of “1” (M = 1.28,t(115) = 6.08, p < 0.001) as shown in Fig. 5.

We also observed a statistically significant effect of the data-to-ink ratio attribute on memorability scores with a “bad” (M = 1.81), i.e., low data-to-ink ratio, being higher than a “good” rating (M = 1.23,t(208) = 6.92, p < 0.001) as shown in Fig. 6. Note that using a corrected t-test, we also arrive at the results that the 3 levels of data-ink ratio are pairwise significantly different from each-other .

Summarizing all of these attribute results: higher memorability scores were correlated with visualizations containing pictograms, more color, low data-to-ink ratios, and high visual densities.

As shown in Fig. 7, diagrams were statistically more memorable than points, bars, lines, and tables. These trends remain observable even when visualizations with pictograms are removed from the data. Other than some minor ranking differences and addition of the map category, the main difference is in the ranking of the table visualization type, which without pictograms becomes least memorable.

The middle panel of Fig. 1 displays the most memorable visualizations that do not contain pictograms. Why are these visualizations more memorable than the ones in the right-most panel?

To start with, qualitatively viewing the most memorable visualizations, most are high contrast. These images also all have more color, a trend quantitatively demonstrated in Sec. 7.2 to be correlated with higher memorability. As compared to the more subdued less memorable visualizations, the more memorable visualizations are easier to see and discriminate as images.

Another possible explanation is that “unique” types of visualizations, such as diagrams, are more memorable than “common” types of visualizations, such as bar charts. This trend is also evident in Fig. 7 in which grid/matrix, trees and networks, and diagrams have the highest memorability scores.

Examples of these unique types of visualizations are each individual and unique, whereas bar charts and line graphs are uniform with limited variability in their visual encoding methodology. Previously it has been shown that an item is more likely to interfere with another item if it has similar category or subordinate category information, but unique exemplars of objects can be encoded in memory quite well [22]. This supports our findings that show high FAR and low HR for table and bar visualizations, which both have very similar visuals within their category (i.e., all the bar charts look alike).

Another possible explanation is that visualizations like bar and line graphs are just not natural. If image memorability is correlated with the ability to recognize natural, or natural looking, objects then people may see diagrams, radial plots, or heat maps as looking more “natural”.

One common visual aspect of the most memorable visualizations is the prevalence of circles and round edges. Previous work has demonstrated that people’s emotions are more positive toward rounded corners than sharp corners [3]. This could possibly support both the trend of circular features in the memorable images as well as the concept of natural-looking visualizations being more memorable since “natural” things tend to be round.

As shown in Fig. 8, regardless of whether the visualizations did or did not include pictograms, the visualization source with the highest memorability score was the infographic category (M = 1.99,t(147) = 5.96, p < 0.001 when compared to the next highest category, scientific publications with M = 1.48), and the visualization source with the lowest memorability score was the government and world organizations category (M = 0.86,t(220) = 8.46, p < 0.001 when compared to the next lowest category, news media with M = 1.46).

This may be a contributing factor to the observed trend (see Fig. 8) that visualization sources that have non-uniform aesthetics tend to have higher memorability scores than sources with uniform aesthetics. This observation refutes our last hypothesis that visualizations from scientific publications are less memorable. This may also be due to the fact that visualizations in scientific publications have a high percentage of diagrams (Fig. 2), similar to the infographic category.

The results of our memorability experiment show that visualizations are intrinsically memorable with consistency across people.

They are less memorable than natural scenes, but similar to images of faces, which may hint at generic, abstract, features of human memory.

Not surprisingly, attributes such as color and the inclusion of a human recognizable object enhance memorability. And similar to previous studies we found that visualizations with low data-to-ink ratios and high visual densities (i.e., more chart junk and “clutter”) were more memorable than minimal, “clean” visualizations.

It appears that we are best at remembering “natural” looking visualizations, as they are similar to scenes, objects, and people, and that pictorial and rounded features help memorability.

More surprisingly, we found that unique visualization types (pictoral, grid/matrix, trees and networks, and diagrams) had significantly higher memorability scores than common graphs (circles, area, points, bars, and lines).