Wang, Q., & Waltman, L. (2016). Large-scale analysis of the accuracy of the journal classification systems of Web of Science and Scopus. Journal of Informetrics, 10(2), 347-364.

Wang, Q., & Waltman, L. (2016). Large-scale analysis of the accuracy of the journal classification systems of Web of Science and Scopus. Journal of Informetrics, 10(2), 347-364.

本研究以引用資料比較Web of Science和Scopus兩個資料庫提供的期刊分類系統(journal classification systems)的正確性。分類系統能應用於各種問題；例如，它可以被用來標定研究區域(Glänzel & Schubert, 2003; Waltman & Van Eck, 2012)，評估和比較研究對各領域的影響(Leydesdorff and Bornmann, 2015; Van Eck, Waltman, Van Raan, Klautz, & Peul, 2013)，以及跨學科的研究(Porter & Rafols, 2009; Porter, Roessner, & Heberger, 2008)。除了Web of Science和Scopus之外，期刊分類系統還有Science-Metrix、NSF(National Science Foundation)分類系統、the UCSD (University of California, San Diego)分類系統以及ANZSRC (Australian and New Zealand Standard Research Classification)，另外Glänzel and Schubert (2003)也提出一個包括期刊和論文的階層式分類系統，以演算法建構的期刊分類方法也有Bassecoulard and Zitt (1999)、Chen (2008)以及 Rafols and Leydesdorff (2009)等，Waltman and Van Eck (2012)的演算法則是以期刊裡出版的論文分類為主。

根據Waltman (2015, Section 3)的文獻分析，在比較Web of Science和Scopus時，主要針對資料庫的覆蓋情形(the coverage of the databases)，例如LópezIllescas, De Moya-Anegón, & Moed (2008)、 Meho & Rogers (2008)、 Mongeon & Paul-Hus (2016)、 Norris & Oppenheim (2007)，或是資料庫用來評估研究產生與影響的準確性，如Archambault, Campbell, Gingras, & Larivière (2009)、Bar-Ilan, Levene, & Lin (2007)、Meho & Rogers (2008)、Meho & Sugimoto (2009)，並未有研究比較與分析它們的分類系統的準確性。

過去Pudovkin and Garfield (2002)曾說明WoS首先利用人工的經驗法則將期刊分配到各類別，之後使用根據引用資料的Hayne-Coulson演算法對新的期刊進行分類。除此之外，Katz and Hicks (1995)、Leydesdorff (2007)、Leydesdorff and Rafols (2009)等研究也曾指出WoS的分類系統是綜合了引用模式、期刊題名與專家意見。但Scopus則未曾有文獻提到其分類系統的建構方式。事實上，在WoS上有兩個分類系統，一個為具有250個類別的類別系統(a system of categories)，另一個則是包含約150個研究領域的研究領域系統(a system of research areas)，此外，另一個分類系統僅包含科學與社會科學，稱為ESI (Essential Science Indicators)。本研究的分析對象是WoS上的類別系統。

Scopus的期刊分類系統則名為ASJC ( All Science Journal Classification)，分為兩個層級，下層有304個類別，上層則分為27個類別。

既然期刊分類系統相當有用，因此有許多研究提出對WoS及Scopus的分類系統進行改善的方法，例如Glänzel等人研究各種方法來驗證與改善WoS的分類系統 (Janssens, Zhang, De Moor, & Glänzel, 2009; Thijs, Zhang, & Glänzel, 2015; Zhang, Janssens, Liang, & Glänzel, 2010)，López-Illescas, Noyons, Visser, De Moya-Anegón, & Moed (2009) 則是對利用WoS分類系統進行的領域劃分，提出改進的方法。在Scopus分類系統的改進方面，則有SCImago團隊(Gómez-Núnez, ˜ Vargas-Quesada, De Moya-Anegón, & Glänzel, 2011, Gómez-Núnez, ˜ Batagelj, Vargas-Quesada, De Moya-Anegón, & Chinchilla-Rodríguez, 2014, Gómez-Núnez, ˜ Vargas-Quesada, & De Moya-Anegón, 2016)。

用來評估期刊分類系統準確性的方法可分為以專家為基礎的方法與書目計量方法(bibliometric approach)。以專家為基礎的方法在遭遇大量資料時有很大的困難，沒有專家有足夠知識來評估所有科學學科上的期刊分類，因此需要相當多的專家加入。書目計量方法可再分為以文本為基礎與以引用為基礎兩種方法，分別以在同一類別下期刊論文的文本相似性與引用模式的相似性大小做為衡量期刊是否應在同一類別的標準。本研究採用的是直接引用(direct citation)關係，先前Klavans and Boyack (2015)曾經利用直接引用關係建構論文的分類系統演算法，他們的結論認為直接引用較書目耦合(bibliographic coupling)或共被引(co-citation)等間接地引用關係更加準確。

綜合以上所述，本研究提出方法的原理可歸納為任一期刊引用該期刊所屬類別下的期刊或被這些期刊所引用的頻次必然較其他類別的期刊之頻次來得高。根據這樣基本原則，本研究制訂兩個檢驗期刊是否被指定於適當類別的標準：
標準1：某一期刊與它所屬類別的其他期刊之間若是只有相當少的引用關係，則這個期刊的分類可能有問題。
標準2：如果某一期刊與其他類別的期刊之間有相當多的引用關係，則這個期刊可能被分到不正確的類別下。

本研究以2010到2014年的WoS及Scopus資料庫上的所有期刊作為分析資料，相關的統計數據如表1所示，比較兩者所收錄的資料，Scopus資料庫除了比WoS更多的期刊種類與類別以外，每一種期刊被指定的類別數目也通常比較多，WoS上的期刊平均被指定到約1.6個類別，但是Scopus則為2.1。


根據第一項標準，WoS及Scopus兩個資料庫上都有許多期刊被指定到不適合的類別上，並且Scopus尤為嚴重，如表3所示。在各個至少有10種期刊的類別當中，選擇至少有一半的期刊符合標準一的類別，這樣的類別，WoS共有17個，Scopus則有高達76個，在兩個資料庫上都出現的類別包括建築學(ARCHITECTURE)、生物物理學(BIOPHYSICS)以及 醫學實驗室技術(MEDICAL LABORATORY TECHNOLOGY)。



兩個資料庫的情況在標準二上都有還不錯的結果(表6)。


同時符合標準一與標準二的期刊，一方面與本身被指定的類別只有較弱的連結，另一方面則與未被指定的類別有較強的連結，分析同時符合標準一與標準二的期刊可以發現，一種可能是在這些期刊上的發表已經和它們的題名和範圍宣告(scope statement)有所差異，另一種可能則是在分類時僅依賴它們的題名。

根據以上的實驗結果，可以歸納以下的幾點結論：
1. 在標準一，WoS的表現比Scopus還要好，因此可以說，Scopus上的期刊通常與它們被指定的類別只有較弱的連結。
2. 在標準二，兩個資料庫的表現都相當好，也就是如果某一期刊與某一個類別的連結較強的話，WoS及Scopus通常會將它指定到這個類別。
3. 整合兩個標準，WoS比Scopus的表現通常要好上許多。

除了上述的結論外，本研究還指出Scopus有些類別有容易混淆的名稱，例如有兩個類別分別命名為LINGUISTICS & LANGUAGE和LANGUAGE & LINGUISTICS，另兩個則為INFORMATION SYSTEMS & MANAGEMENT與MANAGEMENT INFORMATION SYSTEMS。

而且兩個分類系統都缺乏透明性，本研究的作者沒有發現建構與更新分類系統的適當文件。


To examine and compare the accuracy of journal classification systems, we define two criteria on the basis of direct citation relations between journals and categories. We use Criterion I to select journals that have weak connections with their assigned categories, and we use Criterion II to identify journals that are not assigned to categories with which they have strong connections. If a journal satisfies either of the two criteria, we conclude that its assignment to categories may be questionable.

Accordingly, we identify all journals with questionable classifications in Web of Science and Scopus. Furthermore, we perform a more in-depth analysis for the field of Library and Information Science to assess whether our proposed criteria are appropriate and whether they yield meaningful results.

It turns out that according to our citation-based criteria Web of Science performs significantly better than Scopus in terms of the accuracy of its journal classification system.

Classifying journals into research areas is an essential subject for bibliometric studies.

A classification system can assist with various problems; for instance, it can be used to demarcate research areas (e.g., Glänzel & Schubert, 2003; Waltman & Van Eck, 2012), to evaluate and compare the impact of research across scientific fields (e.g., Leydesdorff and Bornmann, 2015; Van Eck, Waltman, Van Raan, Klautz, & Peul, 2013), and to study the interdisciplinarity of research (e.g., Porter & Rafols, 2009; Porter, Roessner, & Heberger, 2008).

Besides the WoS and Scopus classification systems, there are various other multidisciplinary classification systems, for instance the system of Science-Metrix,the system of the National Science Foundation (NSF) in the US,the UCSD classification system, and the system of the Australian and New Zealand Standard Research Classification (ANZSRC).

Science-Metrix assigns “individual journals to single, mutually exclusive categories via a hybrid approach combining algorithmic methods and expert judgment” (Archambault, Beauchesne, & Caruso, 2011, p. 66). The Science-Metrix system includes 176 categories.

The NSF system also offers a mutually exclusive classification of journals, but it is more aggregated, consisting of only 125 categories (Boyack & Klavans, 2014). The system is used in the Science & Engineering Indicators of the NSF.

A more detailed classification system is the so-called University of California, San Diego (UCSD) classification system. This system, which includes more than 500 categories, has been constructed in a largely algorithmic way. The construction of the UCSD classification system is discussed by Börner et al. (2012).

The ANZSRC’s Field of Research (FoR) classification system has a three-level hierarchical structure. Journals are classified at the top level and at the intermediate level. Journals can have multiple classifications.

Furthermore, Glänzel and Schubert (2003) designed a two-level hierarchical classification system, which can be applied at the levels of both journals and publications. They adopted a top-bottom strategy; specifically, they first defined categories on the basis of the experience of bibliometric studies and external experts. They then assigned journals and individual publications to the categories. This classification system has for instance been used for measuring interdisciplinarity. In their analysis of interdisciplinarity, Wang, Thijs, & Glänzel (2015) explain that instead of the WoS subject categories they use the more aggregated classification system developed by Glänzel and Schubert (2003).

Algorithmic approaches to construct classification systems at the level of journals have been studied by for instance Bassecoulard and Zitt (1999), Chen (2008), and Rafols and Leydesdorff (2009).

A more recent development is the algorithmic construction of classification systems at the level of individual publications rather than journals. Waltman and Van Eck (2012) developed a methodology for algorithmically constructing classification systems at the level of individual publications on the basis of citation relations between publications. Their approach has for instance been used in the calculation of field-normalized citation impact indicators (Ruiz-Castillo & Waltman, 2015).

According to a recent literature review (Waltman, 2015, Section 3), previous studies comparing WoS and Scopus are mainly focused on two aspects. One is the coverage of the databases (e.g., LópezIllescas, De Moya-Anegón, & Moed, 2008; Meho & Rogers, 2008; Mongeon & Paul-Hus, 2016; Norris & Oppenheim, 2007) and the other is the accuracy of the databases when used to assess research output and impact at different levels, ranging from individual researchers to departments, institutes, and countries (e.g., Archambault, Campbell, Gingras, & Larivière, 2009; Bar-Ilan, Levene, & Lin, 2007; Meho & Rogers, 2008; Meho & Sugimoto, 2009). However, no study has systematically compared WoS and Scopus in terms of the accuracy of their journal classification systems.

In the case of WoS, Pudovkin and Garfield (2002) have offered a brief description of the way in which categories are constructed. According to Pudovkin and Garfield, when WoS was established, a heuristic and manual method was adopted to assign journals to categories, and after this, the so-called Hayne-Coulson algorithm was used to assign new journals. This algorithm is based on a combination of cited and citing data, but it has never been published.

Besides this, Katz and Hicks (1995), Leydesdorff (2007), and Leydesdorff and Rafols (2009) have indicated that the WoS classification system is based on a comprehensive consideration of citation patterns, titles of journals, and expert opinion.

In the case of Scopus, there seems to be no information at all on the construction of its classification system.

It should be mentioned that in the most recent versions of WoS two classification systems are available, namely a system of categories and a system of research areas.

The system of categories is more detailed. This system, which is the traditional classification system of WoS and the system on which we focus our attention in this paper, consists of around 250 categories and covers the sciences, social sciences, and arts and humanities.

The system of research areas, which has become available in WoS more recently, is less detailed and comprises around 150 areas.

Besides these two systems, Thomson Reuters also has a classification system for its Essential Science Indicators. This system consists of 22 subject areas in the sciences and social sciences. It does not cover the arts and humanities.

The Scopus journal classification system is called the All Science Journal Classification (ASJC). It consists of two levels. The bottom level has 304 categories, which is somewhat more than the about 250 categories in the WoS classification system. The top level includes 27 categories.

The accuracy of a classification system can seriously influence bibliometric studies. For instance, Leydesdorff and Bornmann (2015) investigated the use of the WoS categories for calculating field-normalized citation impact indicators. They focused specifically on two research areas, namely Library and Information Science and Science and Technology Studies. Their conclusion is that “normalizations using (the WoS) categories might seriously harm the quality of the evaluation”.

A similar conclusion was reached by Van Eck et al. (2013) in a study of the use of the WoS categories for calculating field-normalized citation impact indicators in medical research areas.

Glänzel and colleagues have studied several approaches to validate and improve WoS-based classification systems (Janssens, Zhang, De Moor, & Glänzel, 2009; Thijs, Zhang, & Glänzel, 2015; Zhang, Janssens, Liang, & Glänzel, 2010). They have also proposed an improved way of handling publications in multidisciplinary journals (Glänzel, Schubert, & Czerwon, 1999;Glänzel, Schubert, Schoepflin, & Czerwon, 1999).

Related to this, López-Illescas, Noyons, Visser, De Moya-Anegón, & Moed (2009) have studied an approach to improve the field delineation provided by categories in the WoS classification system.

The SCImago research group has made a number of attempts to improve the Scopus classification system (Gómez-Núnez, ˜ Vargas-Quesada, De Moya-Anegón, & Glänzel, 2011, Gómez-Núnez, ˜ Batagelj, Vargas-Quesada, De Moya-Anegón, & Chinchilla-Rodríguez, 2014, Gómez-Núnez, ˜ Vargas-Quesada, & De Moya-Anegón, 2016).

Two types of approaches can be distinguished for assessing the accuracy of journal classification systems. One is the expert-based approach and the other is the bibliometric approach.

Applying the expert-based approach at a large scale is challenging. No expert has sufficient knowledge to assess the classification of journals in all scientific disciplines, so a large number of experts would need to be involved.

In the case of the bibliometric approach, a further distinction can be made between text-based and citation-based approaches.

Text-based approaches could for instance assess whether the textual similarity of publications in journals assigned to the same category is higher than the textual similarity of publications in journals assigned to different categories.

Instead, we take a citation-based approach to assess the accuracy of journal classification systems.

In this paper, we use direct citation relations. This is because “a co-citation or bibliographic coupling relation requires two direct citation relations” (Waltman & Van Eck, 2012, p. 2380), which means that bibliographic coupling and co-citation relations are more indirect signals of the relatedness of journals than direct citation relations.

The use of direct citation relations is also supported by Klavans and Boyack (2015), who study the algorithmic construction of classification systems at the level of individual publications. They conclude that the use of direct citation relations yields more accurate results than the use of bibliographic coupling or co-citation relations.

Thus, the rationale of our approach can be summarized as follows: A journal should cite or be cited by journals within its own category with a high frequency in comparison with journals outside its category.

Based on this basic principle, we define two criteria to identify journals with questionable classifications. One criterion is that if a journal has only a very small number of citation relations with other journals within its own category, then we believe the classification of the journal to be questionable. The other criterion is that if a journal has many citation relations with journals in a category to which the journal itself does not belong, then it seems likely that the journal incorrectly has not been assigned to this category.

We retrieved from the WoS and Scopus databases all journals that have publications between 2010 and 2014....The choice of a five-year time window is a trade-off between on the one hand the stability of journal classification systems and on the other hand the accuracy of our approach based on direct citation relations.

As can be seen in Table 1, the number of Scopus journals included in the analysis is almost twice as large as the number of WoS journals, and Scopus also includes 80 more categories than WoS. Furthermore, although both databases often assign journals to multiple categories, we found that Scopus tends to assign journals to more categories than WoS. WoS assigns journals to at most six categories, whereas in Scopus there turns out to be a journal that is assigned to 27 categories. Additionally, we found that the average number of categories to which journals belong equals 1.6 in WoS and 2.1 in Scopus. This shows that on average journals have significantly more category assignments in Scopus than in WoS.

As can be seen, almost 60% of all journals in WoS belong to only one category, whereas in Scopus more than 60% of all journals are assigned to two or more categories.


WoS has 1390 journals with ti < 100, accounting for 11% of the total number of WoS journals, whereas Scopus has 5808 journals with ti < 100, which is 24% of the total.3 Hence, Scopus has more journals with ti < 100 than WoS not only in an absolute sense but also from a relative point of view.

Taking a further look at Scopus journals with ti < 100, it turns out that they can be roughly divided into three groups. One group consists of arts and humanities journals, another group consists of newly included journals, and a third group consists of non-English language journals.

Table 2 provides some basic statistics on the assignment of journals to categories in WoS and Scopus when journals with ti < 100 and assignments of journals to multidisciplinary categories are excluded. The table shows the number of journals that belong to at least one non-multidisciplinary category and the number of assignments of journals to non-multidisciplinary categories. As can be seen in the table, in the case of Scopus the constraints that we have introduced cause a much larger decrease in the number of journals and the number of journal-category assignments than in the case of WoS.

Table 3 reports for both WoS and Scopus and for three values of the threshold˛the number of journals and the number of journal-category assignments that satisfy Criterion I.

As can be seen, both databases have assigned a significant number of journals to categories that according to Criterion I seem to be inappropriate.

Moreover, no matter which threshold is considered, Scopus performs substantially worse than WoS, not only in the absolute number of journals and journal-category assignments satisfying Criterion I but, more importantly, also in the percentage of journals and journal-category assignments satisfying the criterion.

Next, we identify WoS and Scopus categories with a high percentage of journals satisfying Criterion I. The identified categories may be seen as the most problematic categories in the two databases, because many of the journals belonging to these categories are only weakly connected to each other in terms of citations.

We select categories that include at least 10 journals with ti ≥ 100 and that, for α˛= 0.1, have at least 50% of their journals satisfying Criterion I. The results for WoS and Scopus are reported in Tables 4 and 5, respectively. In the case of WoS 17 categories have been identified, whereas in the case of Scopus 76 categories have been identified, so more than four times as many as in the case of WoS.

There are three categories that have been identified in the case of both databases: ARCHITECTURE, BIOPHYSICS, and MEDICAL LABORATORY TECHNOLOGY.

Table 6 presents for both WoS and Scopus and for five values of the threshold ˇ the number of journals that satisfy Criterion II.

A journal satisfies both Criterion I and Criterion II if on the one hand it has weak connections, in terms of citations, with its assigned categories while on the other hand it has a strong connection with a category to which it is not assigned. More precisely, our focus is on journals for which the current category assignments all satisfy Criterion I, while there is an alternative category assignment that satisfies Criterion II.

Based on the three journals discussed above, we conclude that journals satisfying the combined Criteria I and II can be classified into at least two types. One type refers to journals for which there is a discrepancy between on the one hand their title and their scope statement and on the other hand what they have actually published.  ... The second type refers to journals that seem to have been assigned to a category based only on their title.

First, WoS performs much better than Scopus according to Criterion I. Using the parameter values ˛= 0.05 and ˛= 0.1, the percentage of journals and journal-category assignments satisfying Criterion I is more than two times higher for Scopus than for WoS. Hence, in Scopus journals are assigned to categories with which they are only weakly connected much more frequently than in WoS.

Second, based on Criterion II, WoS and Scopus both perform reasonably well, with WoS having a somewhat better performance than Scopus. For all parameter values that were considered, less than 5% of all journals in WoS and Scopus satisfy Criterion II. In other words, if a journal is strongly connected to a category, WoS and Scopus typically assign the journal to that category.

Third, WoS also presents a significantly better result than Scopus based on the combined Criteria I and II. In WoS there is only one journal satisfying the combined criteria, whereas in Scopus there are 32.

First, Scopus sometimes has confusing category labels. In particular, Scopus sometimes has two categories with very similar labels. Examples are the categories LINGUISTICS & LANGUAGE and LANGUAGE & LINGUISTICS and the categories INFORMATION SYSTEMS & MANAGEMENT and MANAGEMENT INFORMATION SYSTEMS.

Second, lack of transparency is a weakness of both the WoS and the Scopus classification system. We did not find proper documentation of the methods used to construct and update the WoS and Scopus classification systems.

For instance, in the case of a small category, it may be hardly possible for a journal to have a reasonably high relatedness with the category. Therefore it can be expected that many journals belonging to the category will satisfy Criterion I. This may be caused not so much by the misclassification of these journals but more by the small size of the category. On the other hand, in the case of a large category, there may be other problems. A large category may for instance be of a heterogeneous nature and may cover multiple fields that are hardly connected to each other.

Thijs, B., Zhang, L., & Glänzel, W. (2015). Bibliographic coupling and hierarchical clustering for the validation and improvement of subject-classification schemes. Scientometrics, 105(3), 1453-1467.

Thijs, B., Zhang, L., & Glänzel, W. (2015). Bibliographic coupling and hierarchical clustering for the validation and improvement of subject-classification schemes. Scientometrics, 105(3), 1453-1467.

Thijs, B., Zhang, L., & Glänzel, W. (2013, January). Bibliographic coupling and hierarchical clustering for the validation and improvement of subject-classification schemes. In Proceedings of ISSI (pp. 237-249).

本研究利用書目耦合(bibliographic coupling)資訊將收錄於Web of Science資料庫內的期刊分群，利用二次相似性(second order similarities)改善書目耦合資訊產生的相似性矩陣過於疏鬆的問題，再以dendrogram及silhouette測量等資訊決定由Ward的凝聚法(Ward’s agglomeration method)產生集群的數目，最後比較集群結果與Glänzel & Schubert (2003)提出的以期刊為基礎的主題分類架構(the journal-based subject-classification scheme)，了解兩者間的對應情形，並決定集群的命名。

使用書目耦合的優點是因為需要的資料都已呈現在論文或資料庫上，計算論文(即本文所謂的publications)和期刊間的連結不會有延遲，並且建立連結後將會持續保持一致，不隨時間改變。然而，如同其他使用引用資訊的方法，相關的論文或期刊並無法共同具有所有的參考文獻，在相似性矩陣上無可避免地會有大量的0出現(Janssens, 2007; Janssens et al., 2008)，產生極為大量的單一個體(singletons)，並影響後續集群分析的品質。過去解決這個問題的一種做法是將引用文獻與詞語相似性的結果混合，例如Janssens et al. (2008)；另一種的做法是以二次相似性產生相似性矩陣，例如Janssens (2007)、Ahlgren & Colliander (2009)與 Thijs et al. (2013)等研究。

本研究即是利用二次相似性對Web of Science資料庫內的期刊分群，分析的期刊為2006到2009年間出版100筆論文或以上的期刊，共8282種。作法如下：

1. 以Salton提出的餘弦測量法(cosine measure)產出一次相似(first order similarity)，然後以一次相似矩陣再次進行餘弦測量法，產生二次相似性。經過二次相似性的計算後，有10種期刊沒有與其他期刊相連，因此加以移除，因此剩餘的期刊網路上共有8272種。

2. 以Ward的凝聚法產生階層集群，並利用dendrogram及silhouette測量推測可能的集群數目，在本研究由上到下分別有6、14及24種集群數目。silhouette測量方法是對每一種期刊計算一個介於-1和1之間的silhouette數值，正值代表該期刊被分配到適當的集群中。然後將期刊依照其群集分組並以silhouette數值大小排序，產生的圖形可以表示各集群的分群品質，如果在正值的部分有較大的面積，換言之，這個集群具有較多的期刊具有適當的分配，則代表有較好的集群劃分結果。

為了檢驗分群的效果，產生的集群結果與Glänzel & Schubert (2003)提出的以期刊為基礎的主題分類架構進行比較，並且在每一個集群上找出具有代表性的核心期刊(core journals)來分析結果的每個期刊集群。圖五是以網絡來表示集群之間的關係，在圖上可以發現藝術與人文(Arts and Humanities)遠離其他集群，神經科學和行為科學(Neurosciences & Behaviour)介於社會科學和生命科學之間，化學則處於生物科學(Biosciences)、醫學和物理學的中間，較特別的是 一般、區域與社區議題(General,Regional and Community Issues)與生命科學之間有很強的連結。



此外，並且計算14個集群與Glänzel & Schubert (2003)的15個學科(排除Multidiscipline下的期刊)之間的Jaccard指標，呈現為表三。

除了與Glänzel & Schubert (2003)的期刊架構比較以外，本研究也將分群的結果與 ESI (essential science indicators)的類別比較，然而卻發現ESI的劃分與本研究分類結果的結構並不一致。

An attempt is made to apply bibliographic coupling to journal clustering of the complete Web of Science database. Since the sparseness of the underlying similarity matrix proved inappropriate for this exercise, second-order similarities have been used.

Cluster labelling was made on the basis of the about 70 subfields of the Leuven-Budapest subject-classification scheme that also allowed the comparison with the existing two-level journal classification system developed in Leuven. The further comparison with the 22 field classification system of the Essential Science Indicators does, however, reveal larger deviations.

The issue of subject classification and the creation of coherent journal sets has been a major topic in our field since the seventies (see e.g., Narin et al., 1972; Narin, 1976).

The development of computerised methods and the availability of large datasets have shifted the attention from mapping small or single disciplines to the generation of global science maps (Garfield, 1998).

Jarneving (2005) applied bibliographic coupling to map and to analyse the structure of an annual volume of the Science Citation Index.

Janssens et al. (2008; 2009) used a combination of cross-citations and a lexical approach to map journals. Zhang et al. (2010) validated this approach.

The advantage of bibliographic coupling is that there is no delay for the calculation of the link between publications or journals as all data needed are present upon publication or indexing in the database. This also means that link between documents, once established will remain constant over time.

This disadvantage is a result of the very sparse nature of the link matrix (Janssens, 2007; Janssens et al., 2008). The overwhelming number of document pairs does not share any reference at all and thus a large number of zeros occur in the similarity matrix. This deteriorates the quality of the subsequent clustering and may result in an unrealistic large number of singletons (cf. Jarneving, 2005).

As cross-citation data suffers from the same problem, Janssens et al. (2008) introduced a hybrid approach, where they combined citation-based with lexical similarities.

Another solution to overcome the sparseness problem is the use of second order similarities (Janssens, 2007; Ahlgren & Colliander, 2009; Thijs et al., 2013).

A set of journals was compiled from the Web of Science database (SCI-Expanded, SSCI and AHCI). All journals covered in this database between 2006 and 2009 with at least 100 publications in this period are taken into account. This resulted in a set of 8282 journals.

To express the strength of a link between two journals we calculated a first order similarity based on Salton’s cosine measure. The mathematical derivation and interpretation of this similarity measure in the framework of a Boolean vector space model can be found in (Sen & Gan, 1983; Glänzel & Czerwon, 1996).

As bibliographic coupling tends to produce very sparse similarity matrices we applied a second order similarity to reduce this effect. While the first-order similarity is based on the angle between two reference vectors, the second-order similarity is calculated as the cosine of the angle of two vectors holding the first order similarity between two journals.

After the calculation of the second-order similarities, ten journals were removed from the set as they appeared to be singletons without any link to the other journals in the set. The network thus included 8272 journals in total.

Hierarchical clustering with Ward’s agglomeration method was used to create a hard clustering of all the journals.

This method does not provide any automated optimum number of clusters so that the decision was made on the basis of the dendrogram and the silhouette statistics (Rousseeuw, 1987).

Three different levels were chosen. The dendrogram holds strong arguments for a six cluster partitioning while the silhouette plot shows a first peak at 7 clusters. For the highest hierarchical level in the following analysis we use the six cluster solution. At a lower level, the silhouette plot suggests the solutions with 14 and 24 clusters, respectively.

For the evaluation of the specific cluster solution we can rely on the silhouette graphs presented in Figure 4. Each graph presents the silhouette values of the journals in the respective cluster. For each journal a silhouette value is calculated. These values range between 1 and -1 where positive values indicate an appropriate clustering of the journals. Journals are grouped by cluster and ordered from highest silhouette value to lowest. As a consequence the graph gives a good profile of the quality of each cluster. A larger area at the positive side of the vertical axis thus represents a better partitioning.

In order to find an acceptable solution, we decided to use the journal-based subject-classification scheme developed in Leuven (Glänzel & Schubert, 2003). This solution proved most advantageous since both clustering and classification scheme are based on journal assignment. Table 1 presents the hierarchical structure of the three level partitioning. For each cluster the number of journals is mentioned. The labels for the higher levels can be deduced from the lowest level. These labels are taken from the Leuven classification system . The label from the most prominent subject category has been assigned to the corresponding cluster.

Another way to describe the cluster is by using core journals. This notion can be analogously defined as core documents introduced by Glänzel & Czerwon (1996) and extended by Glänzel & Thijs (2011).

In this particular application, a core journal can be identified as journal with at least n links with other journals of at least a given strength r on the second order similarity measure. For the identification of core journals in each cluster we set the number of strong links to at least half the set of journals in the cluster.

As we are using second order similarities this choice is not unreasonable. The value of the strength is chosen such that 12 journals within each cluster comply with both criteria. This means that for more dense clusters the choice of appropriate r-value is higher than in clusters where the journals are not as strongly linked.

Above all, chemistry is at each level a separate cluster. One might expect that at the highest level, chemistry is merged with Physics but we found different patterns.

The second noteworthy observation concerns cluster 17 (Public Health & Nursing). This is a cluster within the ‘Psychology – Neuroscience’ cluster at the highest, six-cluster level. In other partitions or subject classification systems this is attributed to Non-Internal Medicine.

To visualise relations between the 24 clusters we created an additional map. Figure 5 shows these relations.

Despite these multiple assignments we used the Jaccard Index to measure the concordance between the two journal The results are presented in Table 3.

Arts and Humanities is an outlier, Neurosciences & Behaviour acts as a bridge between Social Sciences and Life Sciences, Chemistry takes a central position between Biosciences, Medical Sciences and Physics.Most striking observation in their map is the position of General,Regional and Community Issues which is strongly linked with the Life Science fields.

A 24 cluster solution can be compared with the 22 categories from the classification of Thomson Reuters’ Essential Science Indicators (ESI).

Janssens et al. (2009) showed very low mean silhouette values for the ESI category system in a space with respectively textual distances, cosine similarities of cross-citation vectors and combined distances.

Also in the present study, not all clusters have a unique counterpart in the ESI classification system and vice versa (cf. Janssens et al., 2009). Notably, the ESI fields clinical medicine and engineering, mathematics and social sciences, general are almost uniformly spread over numerous clusters.

Based on this analysis we have to conclude that the segmentation of journals in the ESI categories is not supported by the structure found with bibliographic coupling between journals.

Given this rather weak association between the clustering based on cross-citations and bibliographic coupling, it is a legitimate question to ask which of both methodologies is performing best. A comparison of the mean silhouette values and the silhouette values within each cluster reveals that the methodology presented in this paper results in a more consistent solution.

The 15 cluster solution of cross-citation has a value of 0.04 while the bibliographic coupling results in a value of 0.13.

The main advantage of this method is that clustering can be made as soon as a new database volume is available. The only issue is the lacking cluster labelling that cannot directly be obtained from the method. As a substitute, intellectual classification schemes can be used as reference system. Cluster labelling was made on the basis of the Leuven-Budapest subject-classification scheme that also allowed the comparison with the existing two-level journal classification system developed in Leuven.

The further comparison with the 22 field classification system of the Essential Science Indicators does, however, revealed some striking deviations. These concerned, above all, the fields of clinical medicine, engineering, mathematics and the social sciences. New developments in computer science, neuroscience and psychology as well as in public health (cf. Glänzel & Thijs, 2011) do certainly contribute to such growing deviation.

The main objective of this study was to analyse whether the proposed methodology is appropriate for multi-level journal clustering and to what extent the solutions fit in the framework of traditional subject classification. Further comparison with other solutions such as cross-citation and hybrid methods will be part of future research.