Ошибки в оценке науки, или как правильно использовать библиометрию

Я здесь частично использую материалы моих предыдущих публикаций: Yves Gingras, “Les systèmes d’évaluation de la recherche,” Documentaliste-Science de l’information, 4, novembre 2009, pp. 34–35; “Du mauvais usage de faux indicateurs,” Revue d’histoire moderne et contemporaine, 5 (4 bis), supplt annuel, 2008, pp. 67–79; “Le classement de Shanghai n’est pas scientifique,” La Recherche, mai 2009, pp. 46–50; “Criteria for Evaluating Indicators,” in Blaise Cronin & Cassidy Sugimoto (dir.), Bibliometrics and Beyond: Metrics-Based Evaluation of Scholarly Research (Cambridge, MA, MIT Press, 2014).

См.: Abélard, Universitas calamitatum. Le Livre noir des réformes universitaires, Broissieux, éd. du Croquant, 2003; Christophe Charle et Charles Soulié (dir.), Les Ravages de la «modernisation» universitaire en Europe, Paris, Syllepse, 2007; Franz Schultheis, Marta Roca i Escoda et Paul-Frantz Cousin (dir.), Le Cauchemar de Humboldt. Les réformes de l’enseignement supérieur européen, Paris, Raisons d’agir, 2008; Claire-Akiko Brisset (dir.), L’Université et la recherche en colère. Un mouvement social inédit, Broissieux, éd. du Croquant, 2009.

Информационный доклад от лица комиссии Сената по планированию: Sur le défi des classements dans l’enseignement supérieur. Annexe au procès-verbal de la séance du 2 juillet 2008 [О трудностях рейтингования в высшем образовании. Приложение к стенограмме заседания от 2 июля 2008 года]. Известен под названием «rapport Bourdin» [доклад Бурдена] по имени сенатора Жоэля Бурдена. Доступен на сайте: http://www.senat.fr/rap/r07-442/r07-4421.pdf, цитата со с. 53.

См.: http://www.mines-paristech.fr/Ecole/Classements/.

Доклад Бурдена, там же, с. 98.

См., например, специальные номера по оценке научных исследований следующих журналов: Le Débat, 156, 2009; L’Homme et la Société, 178, 2010; Communication et organisation, 38, 2010; Connexions, 93, 2010; Quaderni, 77, 2012; Esprit, juillet 2012; Mouvements, 71, 2013; см. также: Albert Ogien, Désacraliser le chiffre dans l’évaluation du secteur public, Versailles, éd. Quae, 2013.

Yuri V. Granosky, “Is It Possible to Measure Science? V. V. Nalimov’s Research in Scientometrics,” in Scientometrics 52 (2), 2001, pp. 127–150.

Alan Pritchard, “Statistical Bibliography or Bibliometrics?”, in Journal of Documentation 24, 1969, pp. 348–349.

Alfred J. Lotka, “The frequency distribution of scientific productivity,” in Journal of the Washington Academy of Sciences, 16, 1926, pp. 317–324.

См.: Benoît Godin, “On the origins of bibliometrics,” in Scientometrics, 68, 2006, pp. 109–133.

См.: F. J. Cole & N. B. Eales, “The history of comparative anatomy. Part I: A statistical analysis of the literature,” in Science Progress, 11, 1917, pp. 578–596.

См.: P. L. K. Gross & E. M. Gross, “College libraries and chemical education,” in Science, 66, 1927, pp. 385–389.

См.: Oden E. Shepard, “The chemistry student still needs a reading knowledge of German,” in Journal of Chemical Education, 12, 1935, pp. 472–473.

См.: Richard L. Barrett & Mildred A. Barrett, “Journals most cited by chemists and chemical engineers,” in Journal of Chemical Education, 34, 1957, pp. 35–38.

См.: Robert N. Broadus, “An analysis of literature cited in the American Sociological Review,” in American Sociological Review, 17, 1952, pp. 355–357.

См.: Robert E. Burton & W. W. Kebler, “The ‘half-life’ of some scientific and technical literatures,” in American Documentation, 11, 1960, pp. 18–22.

Eugene Garfield, “Citation indexes for science. A new dimension in documentation through association of ideas,” in Science, 122, 1955, pp. 108–111.

Более подробно о создании ИНЦ см.: Paul Wouters, “The citation culture,” диссертация, Амстердамский университет, 1999, доступна в формате PDF на сайте: http://garfield.library.upenn.edu/wouters/wouters.pdf.

См.: Paul Wouters, “Aux origines de la scientométrie,” in Actes de la recherche en sciences sociales, 164, 2006, pp. 10–21.

См.: Derek J. de Solla Price, “Quantitative measures of the development of science,” in Archives internationales d’histoire des sciences, 4 (14), 1951, pp. 86–93. Два его основных труда: Science since Babylon (New Haven, Yale University Press, 1961) и Little Science, Big Science (New York, Columbia University Press, 1963).

См.: Kenneth O. May, “Quantitative growth of the mathematical literature,” in Science, 154, 1966, pp. 1672–1673.

Derek J. de Solla Price, “Networks of scientific papers,” in Science, 149, 1965, pp. 510–515.

По истории статистики и исследований и разработок (R&D) см.: Benoît Godin, Measurement and Statistics on Science and Technology (London, Routledge, 2005).

ОЭСР (1963), “Méthode type proposée pour les enquêtes sur la recherche et le développement expérimental. La mesure des activité́s scientifiques et techniques,” DAS/PD/62.47, Direction des affaires scientifiques, Paris. В настоящее время этот труд циркулирует в шестом издании.

Derek J. de Solla Price, “The scientific foundations of science policy,” in Nature, 206, 17 avril 1965, pp. 233–238.

Francis Narin, Evaluative Bibliometrics. The Use of Publication and Citation Analysis in the Evaluation of Scientific Activity (Cherry Hill, N. J.: Computer Horizons Inc., 1976).

Ghislaine Filliatreau (dir.), Indicateurs de sciences et de technologies, rapport de l’Observatoire des sciences et techniques (Paris: Économica, 2010).

Yehuda Elkana, Joshua Lederberg, Robert K. Merton, Arnold Thackray & Harriet Zuckerman (éd.), Toward a Metric of Science. The Advent of Science Indicators (New York: John Wiley & Sons, 1978).

Gerald Holton, “Can science be measured?”, in Y. Elkana et al., op. cit., pp. 39–68; Stephen Cole, Jonathan R. Cole & Lorraine Dietrich, “Measuring the cognitive state of scientific disciplines,” idem, ibidem, pp. 209–252; Eugene Garfield, Morton V. Malin et Henry Small, “Citation data as science indicators,” idem, ibidem, pp. 179–207.

См.: Y. Gingras, Sociologie des sciences (Paris: PUF, 2013), pp. 5–6.

См.: T. D. Wilson, “The nonsense of ‘knowledge management’,” Information Research, 8, octobre 2002, article no 144, http://InformationR.net/ir/8–1/paper144.html.

См.: Isabelle Bruno et Emmanuel Didier, Benchmarking: L’État sous pression statistique (Paris: La Découverte, 2013).

См.: Éric Archambault, David Campbell, Yves Gingras & Vincent Larivière, “Comparing bibliometric statistics obtained from the Web of Science and Scopus,” in Journal of the American Society for Information Science and Technology, 60, 2009, pp. 1320–1326.

См.: Keith Pavitt, “Patent statistics as indicators of innovative activities: possibilities and problems,” in Scientometrics, 7, 1985, pp. 77–99.

См.: Francis Narin, Dominic Olivastro, “Linkage between patents and papers: An interim EPO/US comparison,” in Scientometrics, 41, 1998, pp. 51–59.

См. переписку между Гарфилдом и социологом Гарриет Цукерман (Harriet Zuckerman) в: P. Wouters, “The citation culture,” op. cit., p. 102.

Eugene Garfield, Irving H. Sher & Richard J. Torpie, “The use of citation data in writing the history of science,” in Institute for Scientific Information, décembre 1964; Eugene Garfield, “From the science of science to scientometrics: visualizing the history of science with HistCite software,” in Journal of Informetrics, 3, 2009, pp. 173–179.

Henry Small, Physics Citation Index, 1920–1929, Philadelphie, ISI, 2 vols, 1981; “Recapturing physics in the 1920s through citation analysis,” in Czechoslovak Journal of Physics, 36, 1986, pp. 142–147.

http://wokinfo.com/products_tools/backfiles/coss/, http://wokinfo.com/products_tools/backfiles/cos/.

См.: Eugene Garfield & Irving H. Sher, “New factors in the evaluation of scientific literature through citation indexing,” in American Documentation, 14, 1963, pp. 195–201.

См.: Vincent Larivière, Benoît Macaluso, Éric Archambault & Yves Gingras, “Which scientific elites? On the concentration of research funds, publications and citations,” in Research Evaluation, 19, 2010, pp. 45–53.

См.: Alan E. Bayer & John K. Folger, “Some correlates of a citation measure of productivity in science,” in Sociology of Education, 39, 1966, pp. 381–390.

Eugene Garfield, Irving H. Sher, “New tools for improving and evaluating the effectiveness of research,” in M. C. Yovits, D. M. Gilford, R. H. Wilcox, E. Staveley and H. D. Lemer (dir.), Research Program Effectiveness, Proceedings of the Conference Sponsored by the Office of Naval Research Washington, D. C., July 27–29, 1965 (New York: Gordon and Breach, 1966), pp. 135–146. Текст доступен в интернете: http://www.garfield.library.upenn.edu/papers/onrpaper.html.

Jonathan R. Cole, Stephen Cole, Social Stratification in Science (Chicago: University of Chicago Press, 1973).

См.: Perry W. Wilson et Edwin B. Fred, “The growth curve of a scientific literature,” in Scientific Monthly, 41, 1935, pp. 240–250.

См.: Jack H. Westbrook, “Identifying significant research,” in Science, 132, 28 October, 1960, pp. 1229–1234.

J. C. Fisher, “Basic research in industry,” in Science, 129, 19 June, 1959, pp. 1653–1657.

См.: G. Nigel Gilbert, “Measuring the growth of science. A review of indicators of scientific growth,” in Scientometrics, 1, 1978, pp. 9–34.

См.: Francis Narin, “Patent bibliometrics,” in Scientometrics, 30, 1994, pp. 147–155.

См.: P. W. Hart, J. T. Sommerfeld, “Relationship between growth in gross domestic product (GDP) and growth in the chemical engineering literature in five different countries,” Scientometrics, 42, 1998, pp. 299–311.

См.: Vincent Larivière, Éric Archambault, Yves Gingras, Étienne Vignola – Gagné, “The lace of serials in referencing practices: comparing natural sciences and engineering with social sciences and humanities,” in Journal of the American Society for Information Science and Technology, 57, June 2006, pp. 997–1004.

См.: Elisabeth S. Clemens, Walter W. Powell, Kris McIlwaine, Dina Okamoto, “Careers in print: books, journals, and scholarly reputations,” in American Journal of Sociology, 101, 1995, pp. 433–494.

См.: Michael M. Kessler, “Bibliographic coupling between scientific papers,” in American Documentation, 14, 1963, pp. 10–25.

См.: Henry Small, “Co-citation in scientific literature: A new measure of relationship between two documents,” in Journal of the American Society for Information Science, 24, 1973, pp. 265–269.

Этот график в адаптированном виде взят из: Eugene Garfield, “From bibliographic coupling to co-citation analysis via algorithmic historio-ibliography,” см. на сайте http://garfield.library.upenn.edu/papers/drexelbelvergriffith92001.pdf.

См.: Matthew L. Wallace, Yves Gingras et Russell Duhon, “A new approach for detecting scientific specialties from raw cocitation networks,” in Journal of the American Society for Information Science and Technology, 60, 2009, pp. 240–246.

См.: Kevin W. Boyack, Richard Klavans, Katy Börner, “Mapping the backbone of science,” in Scientometrics, 64, 2005, pp. 351–374.

См.: Jean Piaget, “Le système et la classification des sciences,” in Jean Piaget (dir.), Logique et connaissance scientifique (Paris: Gallimard, 1967), pp. 1151–1224.

См. изображения в: Katy Börner, Atlas of science. Visualizing what we know (Cambridge: MIT Press, 2010); а также на веб-сайте: http://scimaps.org/atlas/maps.

Conseil des académies canadiennes [Совет канадских академий], Éclairer les choix en matière de recherche: indicateurs et décisions (Ottawa: Conseil des académies canadiennes, 2012).

См.: Vincent Larivière, Éric Archambault, Yves Gingras, “Long-term variations in the aging of scientific literature: from exponential growth to steady-state science (1900–2004),” Journal of the American Society for Information Science and Technology, 59, 2008, pp. 288–296.

См.: Matthew L. Wallace, Vincent Larivière, Yves Gingras, “Modeling a century of citation distributions,” in Journal of Informetrics, 3, 2009, pp. 296–303.

См.: “Citations track the fate of cold fusion,” in New Scientist, 1713, 21 April 1990.

Anthony F. J. van Raan, “Sleeping beauty in science,” in Scientometrics, 59, 2004, pp. 461–466.

См.: Y. Gingras, “La carrière des publications d’Ettore Majorana. Une étude bibliométrique,” in Revue de synthèse, 134, 2013, pp. 75–87.

Peter A. Lawrence, “The mismeasurement of science,” in Current Biology, 17, 2007, pp. R584–R585.

См.: Y. Gingras, “Revisiting the ‘quiet debut’ of the double helix: a bibliometric and methodological note on the ‘impact’ of scientific publications,” in Journal of the History of Biology, 43, 2010, pp. 159–181.

См.: Y. Gingras, “The collective construction of scientific memory: the Einstein – Poincaré connection and its discontents, 1905–2005,” in History of Science, 46, mars 2008, pp. 75–114; idem, “The transformation of physics from 1900 to 1945,” in Physics in Perspective, 12, 2010, pp. 248–265.

См.: Eugene Garfield, “Random thoughts on citationology. Its theory and practice,” in Scientometrics, 43, 1998, pp. 69–76.

См.: Eugene Garfield, “‘Science Citation Index.’ A new dimension in indexing,” in Science, 144, 8 May, 1964, pp. 649–654.

См.: Norman Kaplan, “The norms of citation behavior: prolegomena to the footnote,” in American Documentation, 16, 1965, pp. 179–184.

Derek de Solla Price, Little Science, Big Science, op. cit., p. 65.

Charles Bazerman, Shaping Written Knowledge (Madison: The University of Wisconsin Press, 1988).

James E. McClellan III, Specialist Control. The Publications Committee of the Académie Royale des Sciences (Paris), 1700–1793 (Philadelphie: American Philosophical Society, 2003), pp. 33 and 38.

См.: Loet Leydesdorff, “Theories of citation?”, in Scientometrics, 43, 1998, pp. 5–25.

См.: Blaise Cronin, The Citation Process. The Role and Significance of Citations in Scientific Communication, Londres, Taylor Graham, 1984.

См.: G. Nigel Gilbert, “Referencing as persuasion,” in Social Studies of Science, 7, 1977, pp. 113–122.

См.: Joseph Bensman, “The aesthetics and politics of footnoting,” in Politics, Culture and Society, 1, 1988, pp. 443–470; в этом длинном эссе нет ни одной ссылки или конкретных данных, а лишь пространная спекуляция на тему прогрессирующей самореферентности и стратегий цитирования.

См.: Vincent Larivière, Alesia Zuccala, Éric Archambault, “The declining scientific impact of theses: implications for electronic thesis and dissertation repositories and graduate studies,” in Scientometrics, 74, 2008, pp. 109–121; см. также: Matthew L. Wallace, Vincent Larivière, Yves Gingras, “A small world of citations? The influence of collaboration networks on citation practices,” in PlosOne, 7, 2012, e33339.doi:10.1371/journal.pone.0033339.

См.: Susan Bonzi, H. W. Snyder, “Motivations for citation: A comparison of self citations and citations to others,” in Scientometrics, 21, 1991, pp. 245–254; см. также: Béatrice Millard, “Les citations scientifiques: des réseaux de références dans des univers de références. L’exemple d’articles de chimie,” in REDES, Revista hispana para el análisis de redes sociales, 19, 2010, pp. 69–93.

См.: ESF [Европейский научный фонд], ESF Survey Analysis Report on Peer Review Practices, ESF, March 2011; www.esf.org/fileadmin/Public_documents/Publications/pr_guide_survey.pdf.

Marie Boas Hall, Henry Oldenburg. Shaping the Royal Society (Oxford: Oxford University Press, 2002), p. 84. Первым научным журналом действительно был Journal des savants, издававшийся Дени де Салло с января 1665 года. Однако три месяца спустя его выпуск был запрещен. Потом он был возобновлен, однако его содержание стало весьма разнородным и касалось всех видов деятельности «литературной республики». Это была скорее литературная газета, нежели реферируемый научный журнал в современном смысле.

Idem, ibidem, pp. 167–169.

См.: Y. Gingras, Sociologie des sciences, op. cit., pp. 48–51.

См.: Lewis Pyenson, “Physical sense in relativity: Max Planck edits the Annalen der Physik, 1906–1918,” in Annalen der Physik, 17, 2008, pp. 176–189.

См. подробнее эту историю в: Daniel Kennefick, “Einstein versus the Physical Review,” in Physics Today, 58, September 2005, pp. 43–48.