Engelhardt, Y. (2007). Syntactic structures in graphics. Computational Visualistics and Picture Morphology, 5, 23-35.

Engelhardt, Y. (2007). Syntactic structures in graphics. Computational Visualistics and Picture Morphology, 5, 23-35.

圖形的目的是用來說明某種的資訊，使得原本無法看見的訊息能被看見 (visualizing the nonvisual)。依據過去的文獻，本研究建議將圖形的建構(building blocks)分為三個部分： a) 顯示的圖形物件 (graphic objects)、b) 配置這些物件使其具有意義來說明訊息的圖形空間 (graphic spaces)以及c) 這些物件的圖形性質 (graphic properties)。在此，圖形物件為一個遞迴的結構 (recursive structure)，也就是：在一個圖形空間上配置的一組圖形物件能夠共同形成一個更高階的圖形物件。

本研究也提出圖形物件的語法類型 (syntactic categories)，用來解釋物件彼此間可以允許的空間關係並且區分圖形的基本構成 (basic constituents)。所有的圖形都是建立在不同語法類型的圖形物件之組合的可能性上，不同語法類型的圖形物件在圖形呈現 (graphic representation)上有不同的行為，此一限制產生它們有不同的空間定位。所有圖形物件的語法類型可分為兩大群組：1) 依附於圖形空間位置上的物件；2) 依附於其他物件上的物件。前者包括節點 (node)、線標示 (line locator)、面標示 (surface locator)以及格標記 (grid marker)；後者則有標籤 (label)、連結線 (connector)、比例區段 (proportional segment)和框架 (frame)。以地圖為例，節點 、線標示、面標示可分別代表地圖上城市、河流與湖泊或國家的標示，格標記則用於表示地圖上的經緯度，標籤則是標明節點代表的城市名稱。各種圖形物件的類型、依附類型與例子，如下表：




根據語言的語法學概念，圖形的語法學 (the syntactics of graphics)研究不同語法類型的圖形物件之間的關係，研究圖形物件與圖形空間之間以規則與限制為基礎的關係，也研究圖形物件如何組合成複合的圖形物件以及複合的圖形物件如何能以更簡單的圖形物件分析。圖形可分為實體場景和物件的影像以及抽象的圖形，前者如照片與地圖等呈現實體的空間，後者則如家族樹 (family trees)、統計圖表 (statistical charts) 等呈現概念性的空間。照片與地圖等利用影像中的空間配置呈現出真實的空間配置；家族樹和圓餅圖 (pie charts) 則以影像中的空間配置呈現非空間性的資訊。然而，實體空間的呈現並不一定表達出被呈現物件的真實座標比例(the true coordinate proportions)，許多圖形也同時由實體和概念性的空間組合而成，例如在地圖上以高度呈現國家的人口密度分布。下表是若干圖形空間的類型與代表。

Building upon the existing literature, we are suggesting to regard the building blocks of all graphics as falling into three main categories: a) the graphic objects that are shown (e.g., a dot, a pictogram, an arrow), b) the meaningful graphic spaces into which these objects are arranged (e.g., a geographic coordinate system, a timeline), and c) the graphic properties of these objects (e.g., their colors, their sizes).

We suggest that graphic objects come in different syntactic categories, such as nodes, labels, frames, links, etc. Such syntactic categories of graphic objects can explain the permissible spatial relationships between objects in a graphic representation.

In addition, syntactic categories provide a criterion for distinguishing meaningful basic constituents of graphics.

It is about images that can be regarded as ‘visualizing the nonvisual’ in an attempt to clarify information of some sort. Such images are often collectively referred to as “graphics”.

In 1914, Willard Brinton writes in his book Graphic methods for presenting facts that “The principles for a grammar of graphic presentation are so simple that a remarkably small number of rules would be sufficient to give a universal language”.

In 1967, Jacques Bertin publishes his classic Sémiologie graphique, in which he analyses the “language” of graphic representations and the “visual variables of the image”.

In 1976, linguist Ann Harleman Stewart examines the properties of diagrams and claims that “Like any language, graphic representation has a vocabulary and a grammar”.

In 1984, Clive Richards proposes a “grammatically-based analysis” of diagrams in his Ph.D. thesis Diagrammatics.

In 1986, Jock Mackinlay suggests that “graphical presentations are actually sentences of graphical languages that have precise syntactic and semantic definitions”. In Mackinlay’s approach, “the syntax of a graphical language is defined to be a set of well-formed graphical sentences”.

In 1987, Fred Lakin publishes his paper “Visual grammars for visual languages”, in which he describes his approach to the “spatial parsing” of graphics, which he defines as “the process of recovering the underlying syntactic structure of a visual communication object from its spatial arrangement”.

Kress and van Leeuwen publish their book Reading images: the grammar of visual design (1996). Unfortunately, it is difficult to extract a systematic approach to a syntactic analysis of graphics from their book.

A paper titled “The visual grammar of information graphics” (1996) by Engelhardt et al., suggests “syntactic categories of visual components”.

Robert Horn, in his book Visual Language (1998), proposes a morphology and a syntax of visual language based partly on the work of Jacques Bertin and on the Gestalt principles of perception.

In his book The grammar of graphics (1999), Leland Wilkinson describes an approach to graphics that is related to object-oriented design in computer science. However, he uses grammatical terminology “metaphorically”, and not in a linguistic sense.

Colin Ware (2000) writes about the “perceptual syntax of diagrams”, describing “the grammar of node-link diagrams” and “the grammar of maps”.

Engelhardt, in his Ph.D. thesis The language of graphics (2002) provides a detailed proposal for the analysis of syntactic structure, which he applies to a broad spectrum of graphic representations.

We propose a notion of graphic objects that will allow for recursive structures: Any graphic representation – and any meaningful visible component of a graphic representation – may be referred to as a graphic object. This means that graphic objects can be distinguished at various levels of a graphic representation. For example, a map or a chart in its entirety is a graphic object. In addition, the various symbols or components that are positioned within that map or chart are graphic objects as well.

A bottom-up description of this principle was given above: a set of graphic objects can be arranged into a graphic space, together forming a single graphic object at a higher level. This “nesting” or “embedding” (Engelhardt 2002) of graphic structures can be referred to as “recursive composition” (Card 2003).

In technical terms, a meaningful graphic space could be defined as a graphic space that involves an interpretation function from spatial positions to one or more domains of information values.

In graphics, not only the possible constituents themselves (graphic objects), and the diverse possible ways of arranging these constituents (in meaningful graphic spaces), but also the possible visual appearances of these constituents (graphic properties such as size, color), could be considered as being part of the graphic “vocabulary”. In this sense we can say that the building blocks of graphics fall into three main categories: graphic objects, meaningful graphic spaces, and graphic properties.



To make a more general statement, we claim that all graphics are based on the possibility of combining graphic constituents (graphic objects) of different syntactic categories (Engelhardt et al. 1996, Engelhardt 2002, 2006).

Graphic objects of different syntactic categories “behave” differently in a graphic representation. The constraints that govern their spatial positioning are different.




All syntactic categories of graphic objects can be divided into two main groups: 1) objects that are attached to locations in graphic space (e.g., node, line locator, surface locator, grid marker are all attached to locations in graphic space), and 2) objects that are attached to other objects (label, connector, proportional segment, frame are all attached to other objects).

Richards (1984) believes that “there seems to be little profit in using such items as an individual dot or line as a unit of analysis. If we are going to use linguistics as a model, then what is needed for present purposes is not the pictorial equivalent of a phoneme or morpheme but something closer to a noun phrase”.

The basic graphic objects in a particular graphic representation are those that can be regarded as functioning in some syntactic category within that particular graphic representation (e.g., as a label, as a node, as a connector, as a proportional segment, etc.).

The distinction between syntactics, semantics, and pragmatics was introduced by Charles Morris (1938, 1946). Morris conceives of syntactics as the investigation of the relationships between signs, of the ways in which complex signs can be constructed from simple ones, as well as the ways in which complex signs can be analyzed into more simple ones (Morris 1946/1971).

The syntactics of graphics investigates the relationships between graphic objects of different syntactic categories. It investigates the rule- and constraint-based relationships between graphic objects (of different syntactic categories) and graphic spaces.

And syntactics investigates how graphic objects can be combined into composite graphic objects, and how composite graphic objects can be analyzed into more simple ones.

Looking at the broad spectrum of graphics we can say that images of physical scenes and objects, such as pictures and maps, represent physical spaces, while many abstract graphics, such as family trees and statistical charts, represent conceptual spaces (Engelhardt 1999, 2002).

In other words, pictures and maps use spatial arrangement in the image to represent spatial arrangement in the world, while family trees and pie charts use spatial arrangement in the image to represent non-spatial information.

Representations of physical spaces do, by the way, not always have to express the true co-ordinate proportions of the represented objects.

Many graphics combine physical and conceptual spaces.

As an example of a true hybrid space (Engelhardt 1999, 2002), think of a three-dimensional landscape drawing of a country in which the drawn “mountains” do not represent physical mountains, but – for example - population density, peaking in the cities and flat in the countryside. In this case, the horizontal plane represents the physical space of the country’s geography, while the vertical dimension represents the conceptual space of population density.




We claim that all types of graphic representation of information can be analyzed in terms of their composition from graphic spaces of different sorts.

We have tried to show that specifying such a visual language means a) specifying the syntactic categories of its graphic objects, plus b) specifying the graphic space in which these graphic objects are positioned, plus c) specifying the visual coding rules that determine the graphic properties of these graphic objects (see table 1).

The syntactic structure of a graphic representation is determined by the rules of attachment for each of the involved syntactic categories (see table 2) and by the structure of the meaningful graphic space that is involved (see table 3).

With this analysis we have attempted to demonstrate that Morris’ original notion of syntactics applies well to the structure of graphics.