Lohse, G. L., Biolsi, K., Walker, N., & Rueter, H. H. (1994). A classification of visual representations. Communications of the ACM, 37(12), 36-49.

Lohse, G. L., Biolsi, K., Walker, N., & Rueter, H. H. (1994). A classification of visual representations. Communications of the ACM, 37(12), 36-49.

information visualization

本研究的目的在於對視覺資訊進行分類，組織這個領域內的系統性研究，提供發展理論所需的概念。對於圖像的分類可分為功能性與結構性兩種，功能性分類著重於圖形的用途與目的，結構性分類則是注重圖形的形式。由於視覺表現是表達知識的資料結構，以知覺推論(perceptual inferences)，取代複雜難困難的認知比較與計算，能夠有助於問題解決與發現，因此本研究將探討結構性分類。過去的分類大多是根據研究者本身的直覺，本研究則是首先讓16位受試者對60種視覺表現進行歸類。程序如下：
1. 每一位受試者對所有的視覺表現，根據本身認為視覺表現間的相似性進行歸類。
2. 以上述的歸類結果，計算每一對受試者之間的Jaccard係數，然後利用完全連接叢集分析(Complete linkage cluster analysis)，找出與其他受試者分類結果最不相同的受試者(outliers)。
3. 刪除分類結果的不相同受試者後，以其他所有受試者的分類結果，建立視覺表現的相似性矩陣，並且進行視覺表現的完全連接叢集分析，進行歸類。歸類結果共計為11個基本類別：圖形(graphs)、表格(tables)、圖像表格(graphical tables)、時間圖表(time charts)、網絡(networks)、結構示意圖(structure diagrams)、程序示意圖(process diagrams)、地圖(maps)、統計地圖(cartograms)、圖標(icons)與照片影像(pictures)。
4. 蒐集受試者對所有視覺表現的評分資料。評分標準包括
-- 空間性(spatial)-非空間性(nonspatial),
-- 非時間性(nontemporal)-時間性(temporal),
-- 難以理解(hard to understand)-容易理解(easy to understand),
-- 具體(concrete)-抽象(abstract),
-- 連續(continuous)-離散(discrete),
-- 吸引人(attractive)-不吸引人(unattractive),
-- 強調整體(emphasizes whole)-強調部分(emphasizes parts),
-- 非數值(nonnumeric)-數值(numeric),
-- 靜態結構(static structure)-動態結構(dynamic process),
-- 包含許多資訊(conveys a lot of information)-包含很少資訊(conveys little information)。
5. 進行主成分分析(principal component analysis, CPA)，嘗試減少評分標準的項目。結果發現這10個評分標準相對獨立並且具有相接近的重要性，因此全數保留。
6. 利用分類與回歸樹(classification and regression trees, CART)建立一個二元分類樹(binary classification tree)藉以決定10個評分標準的評分資料能否預測叢集分析所得到的歸類結果。
7. 最後，以判別分析(discriminant analysis)再次檢驗評分結果和歸類結果的關係。

圖形以幾何物件的位置與大小來表現數量資訊。表格是文字，數字，符號或它們的組合的排列(arrangement)，以緊湊格式呈現出一組事實或關係，可進一步分為數量形表格(numerical table)與圖形表格。比較圖形和表格的呈現方式，圖形強調整體的呈現，而表格著重在部分。時間圖表用以呈現時間性資料。網絡顯示組成份子之間的關係，符號表示組成份子的存在或不存在，以線條、箭頭、接近、相似或包含等方式表示組成份子之間的關係。結構示意圖是物理對象的靜態描述，表達對象的真實座標面向；程序示意圖描述物理對象間動態、連續和時間性的相互關係和程序。比較結構示意圖和地圖與統計地圖，人們可以透過結構示意圖了解性質關係，但從地圖與統計地圖可以得到數量和性質關係。地圖是實際地理的象徵表現，使用符號或文字描繪特定特徵的地理位置，統計地圖則是將數量資料加入地圖上。相較於地圖，使用者較難理解統計地圖。圖標賦予單一的解釋或意義，照片是物件或場景的真實影像。

McCormick, DeFami, and Brown [16] define visualization as “the study of mechanisms in computers and in humans which allow them in concert to perceive, use, and communicate visual information.”

Our research focuses on classifying visual information. Classification lies at the heart of every scientific field. Classifications structure domains of systematic inquiry and provide concepts for developing theories to identify anomalies and to predict future research needs.

Extant taxonomies of graphs and images can be characterized as either functional or structural. 

Functional taxonomies focus on the intended use and purpose of the graphic material. For example, consider the functional classification developed by Macdonald-Ross [14]. ... Other examples of functional classifications can be found in Tufte [22].

A functional classification does not reflect the physical structure of images, nor is it intended to correspond to an underlying representation in memory [1].

In contrast, structural categories are well learned and are derived from exemplar learning. They focus on the form of the image rather than its content. Rankin [18] and Bertin [2] developed such structural categories of graphs.

Rankin used the number of dimensions and graph forms to determine his classification of graph types. Major categories in this scheme include rectilinear cartesian coordinate graphs, polar coordinate graphs, bar graphs, line graphs, matrix diagrams, trilinear charts, response surfaces, topographic charts, and conversion scales.

We adopt the view that visual representations are data structures for expressing knowledge [11, 19].

As such, visual representations can facilitate problem-solving and discovery by providing an efficient structure for expressing the data. Cognitive efficiency results when perceptual inferences replace arduous cognitive comparisons and computations. Since the primary advantage of visual information is that the representation conveys the data structure directly, we chose to develop a structural classification.

Few previous taxonomies and classification schemes for visual representations are based on experimental data; most rely instead simply on the author’s intuitions

Our research focuses on how people classify visual representations into meaningful, hierarchically structured categories.

Our classification (Lohse et al. [12]) was based on subjects’ ratings of the visual similarity between visual representations, and we identified six basic categories of visual representations: graphs, tables, maps, diagrams, networks, and icons.

In addition, we tentatively identified two dimensions that distinguish these clusters. One dimension suggested that a graphic could express either continuous or discrete information, while the second dimension suggested that some visual representations are more efficient than others for conveying information.

The 60 graphical items shown in Figure 1 were used in this study. ... Sixteen subjects were recruited from the students and staff of the University of Michigan.

First, subjects examined all 60 items and named each one to insure that they were familiar with the entire range of items before beginning the rating task. ... Next, subjects rated each of the 60 items on 10 nine-point Likert scales. The 10 rating scales were derived from a frequency analysis of keywords used by subjects to describe each cluster of items during the sorting task of our two previous studies [12, 13].
-- spatial-nonspatial,
-- nontemporal-temporal,
-- hard to understand-easy to understand,
-- concrete-abstract,
-- continuous-discrete,
-- attractive-unattractive,
-- emphasizes whole-emphasizes parts,
-- nonnumeric-numeric,
-- static structure-dynamic process,
-- conveys a lot of information-conveys little information

The final procedure was a bottom-up sorting task, the 60 items were placed randomly on a large table, and the subjects were asked to sort them into groups of similar items. Subjects were given no explicit criteria for judging similarity and could create any number of groups and any number of items per group. Once the subjects had completed their initial groupings, they described each group and explained why all the items in the group were similar. After the experimenter recorded these descriptions, the subjects grouped their initial groupings into higher-order clusters of similar groups. Again, the experimenter recorded the subjects’ explanations of why all the items within a cluster were similar. This process was repeated until all 60 items were placed in a single group.

We first clustered the subjects to determine whether any were obvious outliers among the subjects who sorted the graphics. As a measure of similarity between pairs of subject sorts, we used the Jaccard coefficient [9], which is computed as follows: Jaccard(i, j) = A/(N-B), where A is the number of pairs of graphics in which the members of the pair appear in the same groups for the sortings of both subjects i and j; B is the number of pairs of graphics for which the members of the pair appear in different groups in both subjects' sorts; and N is the total number of graphic pairs or n(n-1)/2 ,where n is the number of graphics.

Complete linkage cluster analysis [10] was then applied to the matrix of Jaccard coefficients. The resulting tree diagram suggested that subject 11 sorted the graphic items in a manner different from the other subjects, and therefore, the data for this subject was removed from all subsequent analyses.

In order to identify groups or clusters of items in the subjects' sortings, a matrix of similarities was constructed by counting the number of times each pair of graphics was grouped together in the subjects’ lowest level sorts. ... The similarity matrix was then used as the basis for complete linkage hierarchical clustering. The resulting tree had nine primary classes or clusters of graphics, two of which had subclasses. These 11 classes are described.

We next sought to determine if the rating scales (that were based on our previous work [12, 13]) were predictive of class membership or clusters derived from the sorting data. To do this, we followed a three-step procedure. First, we used principle components analysis to determine if the 10 scales could be reduced to a smaller set of underlying dimensions. Then we used two different methods for classification: classification trees [3] and discriminant analysis [7]. These techniques were used on the average ratings of the 10 scales for each of the 57 graphical items (i.e., with the three singleton items removed).

A principle components analysis of the data revealed that only one scale, amount of information conveyed, explained less than 9% of the total variance (see Table l). No single scale explained more than 16% of the total variance. The analysis suggests the 10 scales are relatively independent (i.e., nonredundant) and of approximately equal importance (in terms of variance explanation), so we therefore make use of all 10 in the analyses.

The Classification and Regression Trees (CART) methodology [3] was next used to construct a binary classification tree (Figure 3) in order to determine if the ratings on the 10 scales were predictive of membership in the clusters yielded by the hierarchical clustering analysis.

As an additional check on the relationship between the rating scales and the classes derived from the sorting task, we used discriminant analysis. As with the CART methodology, the purpose of the discriminant analysis was to determine the relationship between the rating scales and the sort-derived classes.

Overall, our analyses all provide confirmatory evidence of the taxonomic structure of the graphic items presented in Figure 3. In addition, the results of both the CART analysis and the discriminant analysis suggest that the 10 rating scales can be used as predictors of class membership in the classification.

Eleven categories of visual representations emerged from the classification: graphs, tables, graphical tables, time charts, networks, structure diagrams, process diagrams, maps, cartograms, icons, and pictures. Here we describe these major groups and the type of knowledge conveyed by each class of representation.

Graphs encode quantitative information using position and magnitude of geometric objects. One-, two-, or three-dimensional numerical data is plotted on a Cartesian Coordinate or polar coordinate system. Common graph types include scatterplot, categorical, line, stacked bar, bar, pie, box, fan, response surface, histogram, star, polar coordinate, and Chernoff face graphs.

Graphs emphasize the whole display as compared with tabular data that emphasize parts of the display. ... Tables have less abstract symbolic notation than graphs. 

Tables are an arrangement of words, numbers, signs, or combinations of them to exhibit a set of facts or relationships in a compact format.

Two groups of tables appeared in the classification: graphical and numerical. The primary distinction depended on how numeric information is coded in the table. Graphical tables, like the auto repair records (number 7), used shading to encode frequency of repair data, whereas the statistical table of the critical values of the t statistic (number 21) shows only numeric data. Numerical tables emphasize parts of the whole representation (e.g., individual data values).

Time charts display temporal data. They differ from tables in their emphasis on temporal data.

Network charts show the relationships among components. Symbols indicate the presence or absence of components. Correspondences among the components are shown by lines, arrows, proximity, similarity, or containment. 

There are two types of diagrams, both of which express spatial data.

Structure diagrams are a static description of a physical object. The spatial data expresses the true coordinate dimensions of the object.

Process diagrams describe the interrelationships and processes associated with physical objects. The spatial data expresses dynamic, continuous, or temporal relationships among the objects in process diagrams.

The main difference among similarity measures for maps, cartograms, and structure diagrams is that maps and cartograms express more numeric information than structure diagrams. Thus, people might reason about qualitative relationships from structure diagrams, but reason about qualitative and quantitative relationships from maps and cartograms.

Maps are symbolic representations of physical geography. Maps depict geographic locations of particular features using symbols or lettering.

Maps differ from cartograms in that cartograms super-impose quantitative data over a base map. Therefore, it is not surprising that subjects felt cartograms were more difficult to understand than true maps.

Cartograms are spatial maps that show quantitative data.

Icons impart a single interpretation or meaning for a picture.

Photo-realistic pictures are realistic images of an object or scene. ... Interval properties and distance properties of real world space between objects are preserved in images.

However, subjects in our study characterized photo-realistic images as conveying the least amount of information of all categories in our classification. ... Thus, pictures may contain a great amount of information, but attention must be directed to the visual details of the picture to enable decoding of this information from the picture.

Although photo-realistìc images conveyed less information than all categories in our classification, questions regarding how expert/novice differences influence interpretation also need to be addressed. ... These findings suggest that rather than limit a visualization to an exact copy of a real world object, we can enhance photo-realistic images by enhancing the characteristics of some pixels in the image (smart pixels) to direct and focus our attention to specific information that is relevant to the current task.

Subjects judged cartograms as being hard to understand relative to either maps or graphs. ... As companies develop geographic information systems for superimposing quantitative and spatial information it is important that designers recognize limitations of cartograms for expressing certain types of information and examine alternative visualization tools for expressing such data.

However, it seems more likely that the three-dimensional representations convey more information only to people with an appropriate graph schema for processing information from a novel display format. ... The absence of an accurate diagram schema for displays with unanticipated formats delayed information processing and caused more information processing errors.

Thus, expert-novice differences may not only be a function of graphic arts training but also be a function of having appropriate graph schemata for a particular functional area of expertise.

Wiley [23] found that subjects with graphic arts training remember ordinary pictures better than subjects without graphic arts training, but that memory for unique pictures was consistently high for all subjects regardless of their level of graphic arts training. We might expect to differences between the memory organizations of graph schemata for experts and novices, as novices often lack the necessary schemata to understand the symbolic notation of the graph. However, DcSanctis and jarvenpaa [6] have shown that practice and training can improve the ability to decode information from graphs.

Our classification suggests that network charts present nonspatial information that is difficult to understand. It is important to determine how to present spatial information to facilitate understanding.

Temporal data are more difficult to show in static graphics than cyclic data. Given this limitation of static graphics, it may be important for visualization tools to use dynamic displays or animation for analyzing temporal data. 

Our objectives for developing a classification of visual representations were fivefold:

-- structure systematic inquiry;
-- convey concepts for developing theories;
-- identify anomalies; 

-- predict future research needs;
-- communicate knowledge.

Our classification is subject to four caveats. First, the sample of visual representations influences how well we can generalize our findings. Had we developed our classification from a larger set of items (600 instead of 60), it is not known whether the 10 Likert scales would still characterize all of the items in the classification. Furthermore, we have not identified deep, hierarchical structure within a cluster. For example, what are the major subdivisions within graphs?

Second, the sample of people whose judgments are used to develop the classification must be representative of the entire range of potential users. We have conducted three different experiments using 40 different subjects with a wide range of education, cultural, and graphic arts backgrounds. However, this is still only a small sample from the large population of graph users.

Third, different classification techniques for collecting and analyzing data can and do produce different taxonomies. However, we have used three different techniques over three studies, and each technique has revealed a similar pattern of results.

Finally, our efforts have focused primarily on perceived similarity. We have not investigated whether or not these categories apply to the interpretation of graphics or to the recall of graphical information. For a classification to be useful in both graphical design and research formulation, the classification must represent structure that is used by people in interpreting graphs. This evaluation of our reported classification is our current research goal.