statcheck on the web

To check a PDF, DOCX or HTML file for errors in statistical reporting, upload it below. See the FAQ page for more information about what statcheck can and cannot do.


Settings:

FAQ

statcheck is a "spellchecker" for statistics. It checks whether your p-values match their accompanying test statistic and degrees of freedom.

statcheck searches for null-hypothesis significance test (NHST) in APA style (e.g., t(28) = 2.2, p < .05). It recalculates the p-value using the reported test statistic and degrees of freedom. If the reported and computed p-values don't match, statcheck will flag the result as an error.

statcheck recognizes correlations (r) and t, F, χ2, Z tests and Q tests, as long as they are reported completely (test statistic [a.k.a. test value], degrees of freedom if applicable, and p-value) and in APA Style. Statcheck usually can't find statistics in tables.

statcheck takes into account that test statistics and p values may be exactly (=) or inexactly (< or>) reported. Different spacing has also been taken into account.

statcheck searches for null-hypothesis significance test (NHST) in APA style (e.g., t(28) = 2.2, p < .05). It recalculates the p-value using the reported test statistic and degrees of freedom. If the reported and computed p-values don't match, statcheck will flag the result as an inconsistency. If the reported p-value is statistically significant (α=.05) and the recomputed p-value is not, or vice versa, the result is flagged as a decision inconsistency.

To fix any errors, go to your statistical software to check which of the three numbers (test statistic, degrees of freedom, and/or p-value) you need to correct in your document.

By default, statcheck treats all tests as two-tailed. If you want to take into account one-tailed tests, you can check the box "Correct for one-tailed tests".

When this box is ticked, statcheck will search the entire text for the keywords "one-tailed", "one-sided", and "directional" (taking spacing issues etc. into account). When statcheck finds at least one of those keywords AND an initially inconsistent result would be consistent if it was a one-tailed test, then statcheck treats this case as a one-tailed test and counts it as consistent.

Some common reasons why statcheck doesn't detect some results:

  • The result was not reported according to APA style. This includes minor deviations such as square brackets instead of parentheses, or a semi-colon instead of a comma.
  • The result was not reported completely. statcheck needs three ingredients to detect a result and recalculate the p-value: the reported test statistic, degrees of freedom, and p-value. If one or more of these are missing, statcheck will not pick it up.
  • The result is reported in a table.

Note that a seemingly inconsistent p value can still be correct when we take into account that the test statistic might have been rounded after calculating the corresponding p value. For instance, a reported t-value of 2.35 could correspond to an actual value of 2.345 to 2.354 with a range of p values that can slightly deviate from the recomputed p value. statcheck will not count cases like this as errors.

If you think that statcheck wrongly passed one of your results, please contact us (see below). For more information about statcheck's accuracy, see below.

statcheck flags result as an error when the reported p-value does not match the recalculated p-value. However, there may be cases in which you deliberately reported an inconsistent result. For example, when you conducted a one-tailed test, reported a Bonferroni corrected p-value, or corrected degrees of freedom.

Of course it is also possible that statcheck really made a mistake and erroneously flagged a result as inconsistent. For example, the conversion from PDF (and sometimes also HTML) to plain text and extraction of statistics can result in errors. Some statistical values can be missed, especially if the notation is unconventional.

If you think that statcheck wrongly flagged one of your results, please contact us (see below). For more information about statcheck's accuracy, see the next section.

In typical psychology journals, statcheck detects about 60% of the null hypothesis significance tests. In classifying extracted results as consistent or inconsistent, statcheck has an accuracy between 96.2% and 99.9%, depending on its settings. See Nuijten et al., 2017 for details.

No. Neither the uploaded files nor the results of statcheck are stored anywhere.

See our contact page.

  • The manual: A detailed instruction manual with information on what statcheck can and cannot do, information on how to install and use the statcheck R package, and more.
  • The R package: The R package has additional functionality which allows you to change more settings and to scan entire folders of papers.
  • The paper: The seminal paper in which statcheck was introduced. We ran statcheck on over 30,000 psychology papers and report general inconsistency-prevalences over time and per journal.
  • The validity study: We compared statcheck's performance with manual checks and assessed its accuracy in classifying results as consistent/inconsistent
  • The GitHub page: Here you can find statcheck's latest developments.

Contributors

The statcheck web app is based on the R package statcheck by

  • Michèle Nuijten: author creator
  • Sacha Epskamp: author

statcheck is made possible thanks to the following amazing people!

  • Willem Sleegers: contributor
  • Sean Rife: contributor
  • Paul van der Laken: contributor
  • Edoardo Costantini: contributor
  • Chris Hartgerink: contributor
  • John Sakaluk: contributor

Citation

You can cite statcheck as follows:

Nuijten, M. B. & Epskamp, S. (2024). statcheck: Extract statistics from articles and recompute p-values. R package version 1.5.0. Web implementation at https://statcheck.io.

Software/Packages

Originally, statcheck is written as an R package. The R package statcheck can be found on CRAN. Additional functionality in the package as compared to the web version are for example:

  • scan entire folders of articles
  • changing options, such as the assumed alpha level or whether or not to consider p = .000 as incorrect
  • visualizing the detected results and inconsistencies

The latest updates to the code can be found on statcheck's GitHub page.

Check your manuscript for statistical typos while you're writing.

Try out the beta version!

You can already install and try out the beta version of the statcheck Word add-in! You can find step-by-step installation instructions here.

Contact

For questions about the statcheck Word add-in, contact Willem Sleegers or Michèle Nuijten.

Privacy statement

The statcheck MS Word add-in does not collect or transmit any user information or files. The add-in code is available on GitHub.

Statcheck has been translated into a Python implementation by Hubert Plisiecki. You can find the package on his GitHub page: https://github.com/hplisiecki/statcheck_python. Please note that the Python version is an independent project that may not always be up to date with the latest version of the R version.

Steve Haroz created a web app in which you can let statcheck scan plain text: https://statcheck.steveharoz.com/. Please note that the Python version is an independent project that may not always be up to date with the latest version of the R version.