Skip Navigation
Skip to contents

Perspect Integr Med : Perspectives on Integrative Medicine



Page Path
HOME > Perspect Integr Med > Volume 2(3); 2023 > Article
Does the Chinese Literature Indicate Larger Effect Sizes? This Might Indeed be the Case
Tae-Hun Kim1orcid, Jung Won Kang2orcid
Perspectives on Integrative Medicine 2023;2(3):202-203.
Published online: October 23, 2023

1Korean Medicine Clinical Trial Center, Korean Medicine Hospital, Kyung Hee University, Seoul, Republic of Korea

2Department of Acupuncture & Moxibustion, College of Korean Medicine, Kyung Hee University, Seoul, Repulic of Korea

*Corresponding author: Tae-Hun Kim, Korean Medicine Clinical Trial Center, Korean Medicine Hospital, Kyung Hee University, Seoul, Republic of Korea, Email:
• Received: October 11, 2023   • Revised: October 16, 2023   • Accepted: October 16, 2023

©2023 Jaseng Medical Foundation

This is an open access article under the CC BY-NC license (

  • 21 Download
  • This letter discusses concerns in a currently published meta-epidemiological study on commonly observed large effect sizes in the Chinese literature, focusing on potential selection bias and analysis methods. Researchers should be cautious when conducting systematic reviews that include the literature from specific countries or regions. Regardless of the country, the key issue is to enhance future research quality.
Dear editor,
We have recently read with interest, a meta-epidemiological study on effect sizes in Chinese literature, published in BMC Medical Research Methodology [1]. The issue has long been the subject of debate, addressed by the authors, and has been recognized by scholars as having empirical evidence [2]. Having been involved in complementary and alternative medicine research for several years and having encountered a significant amount of Chinese literature, we have always been mindful of these concerns. We are grateful that this meta-epidemiological study revisited this issue and provided conclusions that support our long-standing observations on this matter. According to the authors’ findings, acupuncture trials reported in Chinese showed significantly larger effect sizes than those in non-Chinese languages (ratio odds ratio 0.51, 95% confidence intervals 0.29–0.91) and they concluded that high risk of bias and study population are significant factors affecting the large effect size [1]. We would like to address and discuss some of these points.
First, the research methodology of this study needs to be revisited. Rather than directly searching for original trials to be included in the analysis, the authors located individual studies from Cochrane reviews, which may have inadvertently introduced selection bias. Depending on author composition and research objectives, Chinese literature may not always be included in Cochrane reviews [3]. According to the authors, only 37% of the 84 systematic reviews included in this study were found in Chinese databases. Thus, there is a considerable possibility of limiting the number of Chinese publications.
Second, we would like to mention the method used to analyze differences between comparison groups. The authors calculated ratio odds ratios to examine differences between articles published in Chinese and non-Chinese languages, or between Chinese and other populations. However, it might have been more appropriate to analyze the impact of publication language and study population on effect size using meta-regression.
Third, the reasoning process leading to the conclusion seems to have some issues. The authors attempted to demonstrate that the language of the trial (Chinese or non-Chinese) is not an influential factor in explaining large effect sizes by comparing studies with a high or unclear risk of bias. However, estimating the actual impact through a comparative analysis of all studies and presenting comparisons between high-risk and low-risk subgroups may provide greater insight. Similarly, in the context of comparing Chinese and non-Chinese populations, a fair interpretation can solely be achieved through the concurrent presentation of subgroup results for studies involving both high and low risk of bias. Using the methodology employed in this study to analyze data could result in a potentially biased perspective. For instance, this methodology does not completely address Chinese-language publications with significant effect sizes and with a low risk of bias. Furthermore, the correlation between Chinese-language publications and effect size in studies of the Chinese population is not covered in the illustrated method. Evidently, issues related to the research question are inadequately covered by the current method.
In conclusion, it is critical to exercise caution when conducting a comprehensive review of the literature, including publications from China. Whether in a positive or negative way, the volume of clinical evidence on acupuncture is significantly influenced by studies conducted and reported from China. Continual endeavors to enhance the caliber of research are imperative, irrespective of its location.
BMC Medical Research Methodology stated that this letter could not be published in its journal. Thus, we submitted this article to Perspectives on Integrative Medicine.

Author Contributions

Conceptualization: THK and JWK. Writing - review and editing: THK and JWK.

Conflicts of Interest

There is no conflict of interest.



Ethical Statement

This letter did not include any personal information. We followed general research ethics guidelines for this letter.

There is no usable data in this article.
  • [1] Li J, Hui X, Yao L, Shi A, Yan P, Yao Y, et al. The relationship of publication language, study population, risk of bias, and treatment effects in acupuncture related systematic reviews: a meta-epidemiologic study. BMC Med Res Methodol 2023;23(1):96. ArticlePubMedPMCPDF
  • [2] Vickers A, Goyal N, Harland R, Rees R. Do certain countries produce only positive results? A systematic review of controlled trials. Control Clin Trials 1998;19(2):159−66.ArticlePubMed
  • [3] Cohen JF, Korevaar DA, Wang J, Spijker R, Bossuyt PM. Should we search Chinese biomedical databases when performing systematic reviews? Syst Rev 2015;4:23. ArticlePubMedPMCPDF

Figure & Data



    Citations to this article as recorded by  

      • PubReader PubReader
      • ePub LinkePub Link
      • Cite
        Download Citation
        Download a citation file in RIS format that can be imported by all major citation management software, including EndNote, ProCite, RefWorks, and Reference Manager.

        • RIS — For EndNote, ProCite, RefWorks, and most other reference management software
        • BibTeX — For JabRef, BibDesk, and other BibTeX-specific software
        • Citation for the content below
        Does the Chinese Literature Indicate Larger Effect Sizes? This Might Indeed be the Case
        Perspect Integr Med. 2023;2(3):202-203.   Published online October 23, 2023
      • XML DownloadXML Download

      Perspect Integr Med : Perspectives on Integrative Medicine