The goal of cognitive metascience is to unite cognitive science, philosophy of science, digital philosophy of science, and digital humanities to explore how researchers reflect on their scientific practice and provide normative guidance during the credibility crisis.

We're using the insights of these large research traditions to study the actual scientific practice. This way, we can enrich considerations required to make valid normative claims that could pave the way towards solving the crisis, especially regarding theory development and assessment.

Here are some resources that we have made available recently.

Here are some corpora we released:

• eLife Open Peer Review Corpus, https://doi.org/10.18150/FKPEQN
• PLoS Open Peer Review Corpus, https://doi.org/10.18150/KZHVGE
• MDPI Open Peer Review Corpus, https://doi.org/10.18150/D5L2EK
• MDPI Open Peer Review Corpus 2 (much bigger), https://doi.org/10.18150/SHKP7B

The MDPI data set in the version 2 is much bigger – the crawler we used was upgraded to find more data. It is probably the biggest open peer review corpus out there. If you are using our datasets, please cite them using the DOI number.

A pure Python open source reimplementation of word sketches (collocation) for corpora of arbitrary size, just the kind you would expect in SketchEngine. However, by using our code, you can:

• ensure reproducibility (anyone can run the same code using your data),

• work with datasets of arbitrary size (even if your institution has very limited access to SketchEngine),

• run word sketches in a Python Notebook (think of Constellate).

• Rorot, W., & Miłkowski, M. (2024). Empirical evidence in conceptual engineering, or the defense of 'predictive understanding'. Forthcoming in Conceptual Engineering. Methodological and Metaphilosophical Issues, Piotr Stalmaszczyk (ed.), Brill / mentis. Preprint: https://doi.org/10.31234/osf.io/a2v4u

In this forthcoming chapter we argue for the use of empirical evidence from digital philosophy in the practice of conceptual engineering, and provide a proof-of-principle by discovering an engineering the concept of 'predictive understanding' hitherto overlooked by philosophers working on understanding, but present in scientists' discourse.

• Miłkowski, M. (2023). Cognitive Metascience: A New Approach to the Study of Theories. Przegląd Psychologiczny (The Review of Psychology), 66(1), 185-207. https://doi.org/10.31648/przegldpsychologiczny.9682

This paper presents the cognitive metascience approach, in particular the account of scientific theories in terms of cognitive artifacts whose contents are not fully accountable for in terms of data. The Polish translation of the paper was published in the same journal: https://doi.org/10.31648/przegldpsychologiczny.9459

• Hensel, W. M., Miłkowski, M., & Nowakowski, P. (2022). Without more theory, psychology will be a headless rider. Behavioral and Brain Sciences, 45, e20. https://doi.org/10.1017/S0140525X21000212

In this commentary to Yarkoni's paper, we argue that psychology needs more explicit theory development, which can start from fairly mundane claims.

• Miłkowski, M. (2022). Turing's Conceptual Engineering. Philosophies, 7(3), 69. https://doi.org/10.3390/philosophies7030069

This paper describes how to intermingle close and distant reading, as applied to Turing's influential 1950 paper on machine intelligence.

• Miłkowski, M. (2022). Cognitive Artifacts and Their Virtues in Scientific Practice, Studies in Grammar, Logic, and Rhetoric 67(3), 219-246. https://doi.org/10.2478/slgr-2022-0012

This paper describes the conceptual foundations of cognitive metascience: scientific representations are cognitive artifacts whose function we should study across multiple dimensions. A short glimpse on the virtues of models through corpora analysis is given.*

• Miłkowski, M., Hensel, W. M., & Hohol, M. (2018). Replicability or reproducibility? On the replication crisis in computational neuroscience and sharing only relevant detail.* Journal of Computational Neuroscience, *45(3), 163--172. https://doi.org/10.1007/s10827-018-0702-z

We claim that low model reproducibility poses more of a threat to long-term scientific progress than low model replicability and reports of computational studies should remain selective and include all and only relevant bits of code.

Rorot, W., & Miłkowski, M. (2024, June 18). How predictive can understanding be in scientific practice? 6th SURe Workshop, London School of Economics, London, UK.

Miłkowski M. (2023, November 20). Empirical evidence in conceptual engineering. Keynote delivered during the Romanian-Polish Workshop in Causation, Understanding, and Knowledge: New Perspectives and Challenges. University of Bucharest, Romanian Academy of Sciences, Institute of Philosophy and Sociology PAS, Bucharest, Romania.

Miłkowski, M. (2023, October 18). Peer Review of Code (in Scientific Research). Keynote presented at the HaPoC-7: 7th International Conference on the History and Philosophy of Computing, Politechnika Warszawska, Polska, Warszawa.

Rorot, W., & Miłkowski, M. (2023, July 11). Distributional semantics in digital philosophy of science: Methodological sketch and a case study Presented at the biennial meeting of the International Society for the History, Philosophy, and Social Studies of Biology, University of Toronto, Canada.

Miłkowski, M. (2023, June 28). The Significance of Theories in Neuroscience: Exploring Their Role in Reproducibility and Discovery. Presented at the Nencki School of Ideas in Neuroscience, Instytut Biologii Doświadczalnej im. M. Nenckiego PAN, Poland, Warszawa. Poland, Warszawa.

Hensel, W., & Miłkowski, M. (2022, October 22*). What Makes Hypotheses Accurate? An Empirical Study. Presented at the Hypothesis in science. *On the 550th anniversary of the birth of Nicolaus Copernicus, Uniwersytet im. Mikołaja Kopernika, Poland, Toruń.

Hvorecky, J., & Miłkowski, M. (2022, August 29). Theoretical Virtues of Cognitive Extension. Presented at the Mind and Technology Congress, Coimbra University, Portugal, Coimbra.

Miłkowski, M. (2021a, December 13). Putting it all together. A plea for theorizing in cognitive (neuro)science. Keynote Address presented at the AISC 2021, 7th Conference of the Italian Association for Cognitive Science, Noto, Italy.

Miłkowski, M. (2022, July 8). From theoretical neuroscience to theories in neuroscience. Presented at the Nencki School of Ideas in Neuroscience, Instytut Biologii Doświadczalnej im. M. Nenckiego PAN, Poland, Warszawa.

Miłkowski, M., Hohol, M., & Hensel, W. (2018, June 22). Replicability of Computational Models: Achilles Heel of Neuroscience. IACAP 2018, IACAP / IFiS PAN, Poland, Warszawa.

Miłkowski, M. (2018, November 25). Replicability or reproducibility? On the replication crisis in computational neuroscience and sharing only relevant detail. Aspects of Neuroscience, Neurobiology Scientific Student Association at the University of Warsaw, Poland, Warszawa.

Wiktor Rorot's project "Investigation of the use of the concept communication in biology and cognitive sciences" has been funded by the National Science Center. Under my supervision, he will study the notion of communication and related notions in life sciences, by relying on digital philosophy of science. The popular description of the project is also available on the funder site.

Are you a student in the Cognitive Science program at the University of Warsaw looking for a new project? Work with me on projects related to cognitive metascience, using language technology and digital philosophy of science. Contact me if you're interested.

Research stays for students in my lab are also available.

We maintain a list of awesome resources for cognitive metascience.