Evaluation of intervention systematic reviews on chronic non-communicable diseases and lifestyle risk factors in low-middle income countries: meta-research

Abstract

Background

Systematic Reviews (SRs) rigorously synthesize findings on a theme, but some articles with this design are redundant due to errors and conflicts. Meta-research aims to rigorously analyze research, assessing SRs’ methodological quality and result reliability. This study evaluates SRs’ overall quality in low- and middle-income countries (LMICs) on chronic non-communicable Diseases (NCDs) and key modifiable risk factors, using assessment tools.

Methods

A search strategy was conducted in the following databases: MEDLINE (via PubMed), Embase, (via Elsevier), Cochrane Library, and Grey Literature for published studies from January 1, 2014 – April 5, 2024. SRs addressing the association between at least one of the four most important modifiable behavioral risk factors (tobacco use, inadequate diet, alcohol consumption, and physical inactivity) and chronic NCDs in populations classified as LMICs according to the ‘World Bank list of countries’ were included. The selected studies were imported into the EndNote 20 software and analyzed using a form for the extraction of their main data and four tools were chosen to assess each of the most important domains of scientific evidence: Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) for article writing; Template for Intervention Description and Replication (TIDieR) for intervention description; A Measurement Tool for Evaluating Systematic Reviews (AMSTAR-2) for methodological assessment; and Grading of Recommendations, Assessment, Development and Evaluation (GRADE) for certainty of evidence.

Results

Nine studies were included in this analysis. The average Overall Score on the PRISMA 2020 checklist was 13.5 for articles published before 2020 and 25.67 for those published after 2020. TIDieR analysis revealed complete correspondence (100%) for item ‘Brief Name’, while other items, like ‘Why’ (89%), and ‘What’, ‘Who Provided’, and ‘How’ (78%), were partially met but significantly so. Regarding AMSTAR-2 criteria, only one study fulfilled all critical items, meeting item 7 by providing a detailed list of excluded studies and justifying each exclusion motive. Additionally, among critical items applicable to multiple articles, only item 11 was consistently fulfilled by all studies. In the final classification, one article achieved a moderate quality rating, three were critically low quality, and five had low quality among the nine evaluated articles. In the GRADE tool evaluation, limitations resulted in estimations for only 19 outcomes and 8 intervention-exposure sets.

Conclusion

The results demonstrated that the writing of recent scientific articles meets most of the PRISMA 2020 criteria, with a checklist being the most used tool. Interventions and exposure were also very well reported, with the TIDieR checklist not being cited in any study as a guiding tool. AMSTAR-2 revealed a methodological approach of varied quality, mainly low and critically low. The GRADE approach classified the certainty of the evidence as generally very low. Therefore, it is necessary to encourage adherence to these approaches to improve the methodological quality in SR studies on chronic NCDs and behavioral factors in LMICs.

Introduction

Chronic Non-Communicable Diseases (NCDs) correspond to a group of pathologies regarded as a new tier of calamity in public health [1]. Among them, we find a vast array of nosological entities capable of affecting all bodily systems, such as cardiovascular diseases, chronic respiratory diseases, gastrointestinal, musculoskeletal, and endocrine-metabolic diseases. Currently, it is estimated that 50% of the global population is affected by at least one chronic NCD, with this group of diseases responsible for about 86% of health expenses worldwide [2].

Many of these pathologies share the commonality of having a direct association between their onset and prevention with human habits and lifestyle, influenced by socio-economic, cultural, sanitary, and educational factors [3]. However, it is worth noting that these diseases disproportionately affect populations in the globe, resulting in epidemiological rates of approximately 75% of chronic NCDs and 82% of premature deaths caused by chronic NCDs in low- and middle-income countries (LMICs) [4, 5]. Furthermore, it is observed that deleterious health habits, like the high prevalence rates of tobacco use, alcohol consumption, inadequate dietary patterns, and physical inactivity status prevail in population groups with low socio-economic status [6,7,8,9]. The situation involving these risk factors is so relevant that the World Health Organization (WHO) estimates that addressing them would decrease the incidence levels of cardiovascular diseases, strokes, and type 2 diabetes mellitus by up to 80% of the global population [5]. Additionally, taking note of this, the United Nations (UN), among its objectives for ‘Sustainable Development’ by the year 2030, has set a goal to reduce premature mortality from chronic NCDs by one-third, based on combating these four entirely modifiable risk factors [10].

In addition, understanding the socio-economic distribution of chronic NCDs in developing countries and their main risk factors requires a high level of confidence in the available studies that investigate the degree of this association between in LMICS. However, given the exuberant number of publications in the biomedical field on this topic [11], healthcare professionals and decision-makers in clinical practice face the daunting task of accessing this information with quality. Thus, SRs are responsible for summarizing findings on a specific topic using a rigorous methodology, making them regarded as the most reliable studies in the scientific evidence landscape by many experts in the field [12, 13]. What many overlook is the fact that, despite their mass proliferation, most of this study design is redundant, contains glaring methodological errors, and serves multiple conflicts of interest [14, 15]. In this perspective, meta-research, whose main objective is to use exhaustive and rigorous techniques to analyze research itself, is a useful study to assess the methodological quality of these designs, gauging the reliability of their results [16]. As material used for this purpose, tools and checklists with the highest degree of scientific foundation have been created to attest to a pristine quality of creation, writing, methodological conduct and results acquisition for SRs.

Therefore, the present meta-research aims to evaluate the quality of SRs which evaluated the association between chronic NCDs and their most significant modifiable behavioral risk factors in populations living in LMICs with scientifically endorsed assessment instruments.

Methods

This study is characterized as a meta-research, a recent study design that seeks to understand how scientific practice is crafted, interpreted, and capable of being produced globally [17]. All of this stems from a thorough analysis of the methodological foundations of other studies, a scenario where research examines its own research methods. As it is classified as a meta-research, in the present study it was not possible to apply writing checklists due to the absence of existing models in the literature.

The methodology of meta-research involves the systematic and critical analysis of scientific studies to evaluate and improve the quality, validity, transparency, and reproducibility of research [18]. This approach begins with the rigorous selection of existing studies using specific criteria and comprehensive databases. The selected studies are then methodologically evaluated using tools such as the Cochrane Risk of Bias Tool, PRISMA guidelines, and AMSTAR-2, which help identify and quantify potential biases, methodological flaws, and transparency gaps. The analysis also includes checking the availability of data and analysis codes, as well as adherence to practices like protocol pre-registration in PROSPERO, which are essential for ensuring the replicability of results [17, 19,20,21]. Beyond its role in identifying and correcting methodological deficiencies in scientific studies, meta-research has significant practical implications by guiding research policies and the development of guidelines aimed at optimizing the use of resources in scientific research [19, 22]. For this analysis, we included secondary studies with the research design of SRs which included Randomized Controlled Trials (RCTs) or Non-Randomized Studies (NRS) from the health literature. The current meta-research is registered on the Open Science Framework (OSF) with a registration protocol at https://doiorg.publicaciones.saludcastillayleon.es/10.17605/OSF.IO/B298V. Only one modification regarding the terminology AMSTAR-2 was made to align with the correct nomenclature of the tool.

Four scientific tools were used to carry out the interpretation of results. The first of them is PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist, published in 2009 with the aim of assisting authors in being transparent and accountable for their reviews, and it has been widely adopted. Recently updated in 2020 to align with advances in systematic review-related terminology and methodology, it now contains 27 great items (disregarding subitems) that describe important aspects for writing the abstract, introduction, methods, results, and discussion¹⁷. This allows for an in-depth understanding of the quality of the SRs’ written report by witnessing the essential items in their writing that are commonly mentioned and those that are absent. Another tool that contributes to ensuring the demonstration of effective interventions in SRs is the Template for Intervention Description and Replication (TIDieR) [23]. Created in 2013, this checklist contains 12 items to improve the quality reporting of interventions by facilitating their structure for authors, aiding reviewers and editors in evaluating descriptions, and making interventions clearer and more replicable for readers [24].

On the other hand, AMSTAR-2 (A Measurement Tool to Assess Systematic Reviews) stands out as a proficient instrument for evaluating the methodological quality of SRs of all study designs. It assesses 16 items, including critical and non-critical domains, judging the methodological quality of SRs based on the consistency and detail of the writing [25,26,27]. Lastly, the GRADE (Grading of Recommendations Assessment, Development and Evaluation) approach is a transparent and universal tool created by the ‘Grade Working Group,’ aiming to create evidence summaries from scientific studies to better guide recommendations for clinical practice [28, 29]. It involves classifying the strength of recommendations and especially assigning the quality of evidence (also known as the level of confidence in the effect estimates) for health study outcomes by a systematic and subjected judgment of several aspects of a SR conduction. Therefore, its use is essential for aiding in the interpretation of the quality of published evidence [30, 31].

PI-ECOT

The PI-ECOT strategy, used to define the study’s central research question, is represented by the following elements:

• Population (P): Men and women living in low-middle income countries (LMICs);

• Intervention-Exposure (I-E): Smoking, alcohol consumption, imbalanced diet, or physical inactivity;

• Comparison (C): Absence of smoking, alcohol consumption, imbalanced diet, or physical inactivity;

• Outcome (O): Primary outcomes - evaluate the quality of writing structure, methodological conduct, reporting mode of interventions or exposures and the certainty of confidence of SRs that investigated the association of Chronic Non-Communicable Diseases (NCDs) and behavioral risk factors in LMICs according every single item of PRISMA, AMSTAR-2, TIDieR and GRADE tools. Secondary outcomes - summarize possible described effects in SR that arose from each intervention for its specific risk factor analyzed in each study, such as economic impact, reduction in hospitalizations, or improvement in well-being, quality of life, education resilience, social, physical, and occupational functioning to better understand the scenario of intervention delivered.

• Time (T): Between the years 2014 to 2024.

Inclusion and exclusion criteria

The following eligibility criteria for inclusion was: SRs addressing the association between at least one of their four most important modifiable behavioral risk factors (tobacco use, inadequate diet, alcohol consumption, and physical inactivity) and chronic NCDs in populations from low-middle income countries according to the ‘World Bank list of countries’ (2019) [32]. For this, different means of intervention (e.g., online, by phone, in person), by different professionals (e.g., multidisciplinary health team, teachers), and in different sectors (e.g., primary, secondary, and tertiary care, health education) were considered. The temporal cut-off encompassed studies published in the last seven years (January 1, 2014 - December 31, 2021), a timeframe sufficient to recognize the implementation of the GRADE (established in 2000), AMSTAR-2 (established in 2007), PRISMA (established in 2009), and TIDieR (established in 2013) tools, with the minimum cut-off year determined as 2014 (one year after the creation of the TIDieR tool).

Furthermore, there were no restrictions regarding the language of the article, utilizing digital translation strategies for any study accessed in a language other than Portuguese, English, or Spanish. Studies were excluded if they met at least one of the following criteria: (1) not being a SR; (2) the absence of lifestyle risk factors as intervention or exposure; (3) not including at least one NCDs as an outcome; (4) the population originating from high-income countries (HICs).

Electronic searches

The databases used were MEDLINE (via PubMed), Embase (via Elsevier), Cochrane Library, as well as Grey Literature (http://www.greynet.org/opengreyrepository.html), with searches dated from January, 2014 to April 5, 2024. The search strategies used are present in Supplementary File 1 (Search Strategy), which was created separately due to the large number of terms employed and the magnitude of the compiled and applied search strategies. All of them were based on Medical Subject Headings (MeSH) terms related to the components of the PI-ECOT strategy. Manual searches were not conducted, as provided for by our protocol, and Grey Literature was carried out with the same search strategy for MEDLINE.

Study selection

The search was conducted by two independent reviewers (GDB and GFMVS). Upon obtaining all the studies, there was firstly the removal of duplicates. Then, the reviewers read the titles and abstracts and classified them as “yes,” (article meet inclusion criteria) “no,” (article does not meet inclusion criteria or does meet exclusion criteria) or “maybe” (article apparently meet inclusion criteria) in a blinded and independent manner using the free web-based platform Rayyan, designed for organizing articles to construct a SR. At the end of the selection process, the blinding of the platform was removed, and consensus was subsequently reached to resolve any discrepancies in the items assigned by each reviewer to the evaluated articles. Those finally classified as “no” were excluded, while those marked as “yes” or “maybe” were read in full and assessed against the inclusion and exclusion criteria after complete reading. An experienced reviewer (AJG) resolved any disagreements not resolved through consensus, providing the final verdict. A comprehensive list of reasons for study exclusions is available in a supplementary file.

Data extraction

All studies included were imported into the EndNote 20 software. A randomization process of the selected SR (calculated N) was carried out on the “Randomizer” website, generating a sequence of numbers to provide a completely random reading and data extraction order of the articles evaluated. The EndNote 20 references were then separated for full-text reading and application of the approaches.

Data extraction was carried out using PRISMA, AMSTAR-2, TIDieR, and GRADE tools in the included studies, integrally. All of these approaches have a high degree of scientific validation and are recommended for the integrated assessment of SRs [23, 25, 28, 33, 34], each one corresponding to a crucial domain for the construction of this research design.

Two reviewers (GFMVS and BHSC) independently extracted data, and for the identified primary outcomes, they applied checklists from the PRISMA (quality and bias in reporting systematic reviews), TIDieR (quality of reporting and bias in intervention descriptions in systematic reviews), and AMSTAR-2 (quality and bias in systematic review methodology) tools, respectively. On the other hand, one reviewer (GDB) also independently applied the GRADE tool (level of certainty and biases in the body of evidence).

Furthermore, an extraction data form was created by authors to assess the main data of included studies plotting the data in a Microsoft Excel spreadsheet for a transparent and objective analysis. The extraction sheet included the following domains:

• (1) Study details: objective, project details, country where the study was conducted, funding, details about the intervention delivery location (i.e., city or community), target condition or risk factor (e.g., subliminal symptoms, coexistence of deleterious lifestyle habits) and the article’s self-reported use of at least one SR quality assessment tool (i.e., PRISMA, TIDieR or GRADE).

• (2) Participants: sample size (intervention and control groups at baseline and follow-up), sociodemographic characteristics (e.g., age and gender).

• (3) Interventions/Exposures: description of the intervention/exposure including frequency and duration, number of sessions, format (e.g., one-on-one or group), intervention cost.

• (4) Delivery of the intervention/exposure: setting where the intervention/exposure took place (e.g., school, home, health clinic), who delivered the intervention (e.g., doctor, nurse, psychologist, teacher, lay health worker, etc.), and whether it was delivered by an individual practitioner or a team of individuals, if there was intersectoral collaboration (e.g., between health and education or child protective services).

• (5) Comparison Groups: characteristics and procedures for the selection of comparison groups (e.g., matching vs. randomization).

• (6) Results: effect obtained its size, direction, and confidence interval. When numerical data were not presented, the outcomes were summarized narratively.

For the analysis of writing, the PRISMA checklist, versions 2009 and 2020, was used since some articles were published before the release of the new PRISMA 2020 version.

The PRISMA 2009 statement consists of a 27-item checklist used to assist authors in providing a transparent report of SRs and meta-analyses. Of the 27 items, one assesses the reporting of the title, one assesses the abstract, two assess the introduction, twelve assess the methods, seven assess the results, three assess the discussion, and one assesses the reporting of funding. The article by Moher et al. [33], providing the explanation and elaboration of the PRISMA 2009 checklist, was consulted to assess whether each item was informed appropriately.

The PRISMA statement criteria’s can be accessed in Supplementary File 2.1 (Checklist 2009) and 2.2 (Checklist 2020). The reporting of every single item in the included studies was classified as: ‘Informed’ (I), ‘Partially Informed’ (PI), ‘Not Informed’ (NI), and ‘Not Applicable’ (NA). In PRISMA 2009, items 16 (‘Additional Analyses’), 21 (‘Synthesis of Results’), and 23 (‘Additional Analysis’) were classified as NA if the review involved a Meta-Analysis. Thus, in PRISMA 2020, items 13e (‘Synthesis methods’) and 20c (‘Results of syntheses’) were also considered NA in articles that did not plot a Meta-Analysis. Writing items informed in an appendix or properly referenced protocol were also evaluated.

The PRISMA 2020 statement is an update of the PRISMA 2009 statement. The PRISMA 2020 statement consists of a main checklist and an abstract checklist. The main checklist has 27 great items: one assesses the reporting of the title, one assesses the reporting of the abstract, two assess the introduction, eleven assess the methods, seven assess the results, one assesses the discussion, and four assess other information. The abstract checklist has 12 items and is actually a subdivision of the great item 2 of the main checklist. In total, the checklist has 53 topics.

Unlike the PRISMA 2009 statement, the PRISMA 2020 statement included some subdivisions items such as items 10 (‘Data items’), 13 (‘Synthesis methods’), 16 (‘Study selection ’), 20 (‘Results of syntheses’), 23 (‘Discussion ’), and 24 (‘Registration and protocol’). The article by Page et al. [34]., providing the explanation and elaboration of the PRISMA 2020 checklist, was consulted to assess whether each item was reported appropriately.

An overall point’s score of related items from the PRISMA checklist, created by simple and weighted average, were presented in bar charts to facilitate visualization as an additional analysis. The maximum Total PRISMA 2020 Score of an article that did not conduct meta-analysis and an article that conducted meta-analysis are respectively: 51 and 53 (quoting great items and sub-items). The maximum Total PRISMA 2009 Score of an article that did not conduct meta-analysis and an article that conducted meta-analysis are respectively: 24 and 27.

The results from the PRISMA 2009 and PRISMA 2020 statements were also synthesized in individual tables for both statements obtained from the PRISMA website. All tables for individual studies are available in Supplementary Files 3.1 and 3.2 (Individual Study Tables). Each item was classified as ‘Informed’, ‘Partially Informed’, ‘Not Informed’, or ‘Not Applicable’, with location of all information identified in the individual tables.

The AMSTAR-2 tool is used to assess the methodological quality of SRs, comprising 16 items, among which seven domains are critical, and nine are non-critical [25,26,27]. The quality of the review can be classified based on the fulfilment or non-fulfilment of these items as ‘Critically Low’, ‘Low’, ‘Moderate’, and ‘High’. ‘High’ was considered if no critical domain was identified. ‘Moderate’ if it fails in more than one non-critical domain. ‘Low’ if it fails in a critical domain (with or without failures in non-critical items). ‘Critically Low’ if more than one critical item is not fulfilled. The results of the articles’ assessment using this tool were presented through a table where the 16 items of the tool were evaluated as ‘Yes,’ ‘Partial Yes,’ ‘No,’ and ‘No meta-analysis conducted,’ with critical items marked with an ‘*’. Based on the fulfillment or non-fulfillment of these items, the articles were classified into ‘High’, ‘Moderate’, ‘Low’, or ‘Critically Low’ methodological quality.

The capable tool of ensuring the demonstration of interventions quality report was the Template for Intervention Description and Replication (TIDieR). The checklist contains 12 items, including the brief name of the intervention/exposure, why it was performed, what it is (materials and procedures involved), who performed it, how it was performed, where it was performed, when it was performed, the costs involved, any customizations, modifications, and the satisfaction level when employed [23, 24]. The results obtained by the TIDieR tool were classified as ‘Informed’ or ‘Not Informed’ based on whether the tool items were fulfilled or not [24]. At the end of the table, a column called ‘Number of Studies (Percent)’ was inserted with the absolute and percentage numbers of the corresponding items in each of the studies evaluated. The score of each article regarding the TIDieR items considered ‘Informed’ and those considered ‘Not Informed’ was also presented in bar charts for easier visualization. All tables for individual studies are available in Supplementary File 4.

The GRADE approach constitutes a well-structured and transparent approach with its central goal being to assign the quality of evidence for study outcomes [28]. This is achieved through a final classification into the domains ‘Very Low’, ‘Low’, ‘Moderate’, and ‘High’. This grading is based on a final consensus judgment involving the downgrading of five domains that decrease quality (Risk of Bias, Inconsistency, Indirectness, Imprecision, and Publication Bias) and three domains that increase quality (Large Effect Size, Dose Response Gradient, and Possible Confounding Factors) [29].

The results from GRADE, in addition to being represented in a synthesized table format, were also allocated in a Summary of Findings (SoF) – a table created in the free web-based application Guideline Development Tool and recommended by GRADE handbook [28] – to evaluate the certainty evidence into 5 domains: ‘Unmeasured’, ‘Very Low’, ‘Low’, ‘Moderate’, and ‘High’. The SoF table is part of the GRADE approach application routine. Its purpose is to facilitate the determination of outcome types and the certainty of evidence by making all assessment items for an outcome visible and tabulated. Each of these aspects was also presented with their respective effect measures as found in the assessment, including Percentage of Prevalence (%), Odds Ratio (OR), Mean Difference (MD), and Standardized Mean Difference (SMD). As no meta-analysis was performed in our study, no heterogeneity strategy was employed.

In the approach to recover possible missing data (i.e., methods, outcomes, supplementary material etc.) two emails (one for notification and another as a reminder) were programmed to be sent within a 7-day interval to the authors. However, there was no need to request missing data because no missing information was identified.

Results

Identification and eligibility of studies

A total of 360 articles were obtained from the search strategies. They were input into the Rayyan platform for initial assessment. We identified 9 duplicated articles, which were excluded from the investigative process, resulting in 351 unique studies. These studies were initially assessed based on their titles and abstracts to determine the relevance of analysis, leading to the exclusion of 335 of them and the selection of 16 for further investigation. During the full-text reading, the 16 studies were reduced to 9, thus forming the final included studies for the analysis of this research. The number of excluded studies is outlined in the selection flowchart found in Fig. 1 (PRISMA Flowchart). A comprehensive list of all studies and their respective reasons for exclusion is available in Supplementary File 5 (Exclusion Criteria).

Fig. 1

PRISMA flowchart

Study descriptions

At the end of the study selection process for evaluation, 9 SRs with or without meta-analysis met the inclusion criteria. The main characteristics of each included article are presented in Supplementary File 6 (Extraction Table). We also provide a summarized and informative representation of the results obtained through the application of the PRISMA (Supplementary Files 2.1, 2.2, 3.1 and 3.2), TIDieR (Supplementary File 4) and GRADE (Supplementary File 7) tools, respectively. Based on this, a total of 40 health outcomes were obtained using the GRADE approach and interpreting the main characteristics of studies. Every one of them describes one chronic disease associated with at least one modifiable behavioral risk factor (smoking, alcohol consumption, sedentary behavior, or unbalanced diet) in populations from LMICs. The publication dates of the 9 reviewed SRs corresponded to the years 2016³⁶,2017³⁷, 2018^38,39,40,41,2021^42,43 and 2022⁴⁴, with authors affiliated with institutions across all five continents, including countries such as China [43],India [36], Ethiopia [41],South Africa [39, 42],Pakistan [35], Japan [40], Nigeria [39], Nicaragua [37], Jamaica [41], Barbados [38], Netherlands [41], and London [38, 39, 41]. Only 2 SRs analyzed only RCTs [36, 43], while the rest included both RCTs and NRSs [35, 39, 40], or only NRSs [37, 38, 41, 42]. This highlights how the majority of evidence was generated by lead authors affiliated with institutions in LMICs, while most articles are supported by co-authorship of writers from HICs.

PRISMA checklists assessment

Despite only 3 articles being published after the release of the new PRISMA 2020 checklist [41,42,43], both versions of the checklist (PRISMA 2009 and PRISMA 2020) were applied to all articles included in this meta-research.

From the sum of the number of items ‘Informed’ (I), ‘Partially Informed’ (PI), ‘Not Informed’ (NI), and ‘Not Applicable’ (NA), an Overall Score of the studies was obtained to the PRISMA 2009 and PRISMA 2020. The maximum score in PRISMA 2020 if all the items was considered ‘Informed’ is 42. The maximum score in PRISMA 2009 if all the items was considered ‘Informed’ is 27. The average of Overall Score of items ‘Informed’ satisfactorily on the PRISMA 2020 checklist for articles published before 2020 was 13.5. For articles published after 2020, the average was 25.67, as shown in Fig. 2 (Overall Score PRISMA 2020).

Additionally, only 3 articles did not mention the use of the PRISMA checklist during the reporting [35, 37, 43], as presented in the supplementary files 2.1, 2.2, 3.1 and 3.2. The overall PRISMA Score among these articles was considerably lower than the others, as reported in Fig. 3 (Overall Score PRISMA 2009).

Fig. 2

Overall Score items of checklist PRISMA 2020. (Σ = summation)

Fig. 3

Overall Score items of checklist PRISMA 2009. (Σ = summation)

The average General PRISMA 2009 Score, as shown in Fig. 3 (Overall Score PRISMA 2009), was higher among articles published after the creation of the PRISMA 2020 version [41,42,43].

TIDieR assessment

In general, there was a high level of compatibility between the included articles and tool items. Among the 12 recommended items, item number 1 (‘Brief Name’) achieved complete correspondence (9/9–100%), while all others were partially filled, such as item 2 (‘Why’) (8/9–89%) and items 4 (‘What’), 6 (‘How’), and 7 (‘Where’) with the same degree of compatibility (7/9–78%). Additionally, items 8 (‘When and How Much’) (6/9–67%) and 5 (‘Who Provided’) demonstrated intermediate correspondences (5/9–56%). Finally, some requirements of the tool were observed to a limited extent in the SRs, with small percentages (2/9–23%) for items 11 and 12 (‘How Well’) and the absence of compliance (0/9) in items 9 (‘Tailoring’) and 10 (‘Modifications’). It is worth mentioning that these two last items described refer to unusual aspects in most interventions performed. The explanation is based on the fact that not all of them are capable of being personalized, qualitatively evaluated or even undergoing changes during their application. Therefore, interpreting the studies in a personalized manner, the low or absent level of compatibility of these items may have little impact on the quality of the reporting of these interventions. Figure 4 (Overall Score TIDieR) graphically illustrates the absolute numbers of ‘Informed’ or ‘Not Informed’ items of the included studies and the requirements outlined by the TIDieR tool.

Fig. 4

Overall Score TIDieR. (Σ = summation)

AMSTAR-2 assessment

The results of the 9 articles for each of the domains of AMSTAR-2 are presented as “Yes,” “Partial Yes,” “No meta-analysis,” and “No” in Fig. 5 (AMSTAR-2 assessment of methodological quality). Also included in this table is the methodological quality based on the fulfillment or non-fulfillment of critical items and the outcome of a chronic NCD evaluated by each article. Nearly half of the SRs did not conduct a meta-analysis. Whether or not meta-analyses are performed plays a significant role in the methodological evaluation of studies, directly impacting the scores in specific AMSTAR-2 domains. However, it is important to emphasize that the absence of a meta-analysis, in itself, does not constitute a critical evaluation criterion, unless this absence compromises the robustness of the methods used to synthesize the evidence.

Fig. 5

AMSTAR-2 assessment of methodological quality and DNCT evaluated by each article. CL = critically low. L = low. M = moderate. NA = Not applicable (no meta-analysis was conducted). *= Items marked with this symbol correspond to the AMSTAR-2 critical domains

Fig. 6

Overall Score AMSTAR-2

Individually analysing the items of AMSTAR-2, among the critical items, item 2 reveals that 3 out of 9 articles failed to record a protocol and justify any changes during the study. Item 4 was partially addressed by 5 studies that failed to conduct and/or describe the search strategy in detail. Only one article [31] fulfilled item 7 by providing a list of excluded studies and justifying the exclusions. Item 9 was not fulfilled by 1 article that did not use a satisfactory technique to assess bias in the included studies. Item 11 was fulfilled by all applicable articles, and item 13, regarding considering bias in included studies when discussing results, was not fulfilled by one study. Item 15, concerning investigating publication bias of included articles, was not fulfilled by one applicable article, and 4 other studies did not conduct a meta-analysis, making items 11, 12, and 15 not applicable.

A failure in a single critical item is sufficient for classifying the SR as having low methodological quality. Since only one review met the criteria of item 7, out of the 9 articles evaluated, only one was considered to have moderate quality, three were critically low, and five were of low quality. Figure 6 illustrates the distribution of studies across the AMSTAR-2 assessment categories: Low, Critically Low, and Moderate.

GRADE approach

Outlining outcomes

The approach generated 40 health outcomes, all of them relevant to one chronic disease associated with at least one modifiable behavioral risk factor (smoking, alcohol consumption, sedentary lifestyle, or unbalanced diet) in populations of LMICs. As a wide variety of designs were included within SRs— only randomized controlled trials (RCTs), only non-randomized studies (NRSs), or both — interventions and exposures were compared with risk factor and obtained the following data: 6 comparisons for unbalanced diet [36, 38, 40,41,42,43];4 comparisons for decreased or nonexistent physical activity [36, 38,39,40];2 comparisons for smoking [36, 38]; and 2 comparisons for alcohol consumption [36, 37]. Except one study [35] did not declare the dietary exposure patterns assessed.

All outcomes are related to chronic NCDs, specifically hypertensive disorders (5 outcomes), cardiac structure and function diseases (2 outcomes), pathological eating habits (1 outcome), glycemic profile increased (9 outcomes), dyslipidemia (7 outcomes), weight gain (6 outcomes), low body weight status (3 outcomes), cervical cancer (1 outcome), metabolic syndrome (1 outcome), chronic kidney disease (1 outcome), smoking (1 outcome), sedentary lifestyle (2 outcomes), and acute respiratory infections (1 outcome). Of these, 21 outcomes could not have their effects measured and certainty estimated, as they were narrative and not grouped outcomes. Consequently, among all 13 interventions-exposures conducted, 5 could not be investigated either. Therefore, only 19 of the results and 8 of the intervention-exposure sets were estimated by the GRADE tool, all of them derived from SRs that included only RCTs or only NRSs studies. The detailed reasons for understanding the decrease and increase in the certainty level of the evidence can be accessed directly in Supplementary File 7 (Summary Table of GRADE Results).

Certainty of evidence

From the 40 primary outcomes, only 19 were able to undergo the assessment of the quality level using the GRADE tool. This limitation arose due to the implementation of a narrative synthesis description of the results in some articles [35, 39,40,41], rendering the application of the GRADE assessment criteria for determining the degree of confidence unfeasible. Only two certainty of evidence categories were obtained through the final application of the approach: ‘very low’ (89.5%) and ‘moderate’ (10.5%). The only outcomes classified as ‘moderate’ were derived from a SR with only RCTs, presenting one residual outcome classified as ‘very low’ [43]. In turn, all remaining outcomes categorized as ‘very low’ originate from SRs with only RCTs and SRs with only NRSs. Furthermore, due to the inability to apply the tool to the presentation format of certain outcomes, health risk factors were also unable to be analyzed. A total of 13 studies conducted comparisons, 8 of them could have their evidence truly tested. Among them, 3 comparisons related to dietary factors; 2 comparisons regarding the degree of physical inactivity; 2 comparisons regarding smoking habits; and a sole comparison concerning excessive alcohol consumption.

It should also be mentioned that the application of the tool was conducted in only two articles [36, 42], both from their original version, and their estimated levels of evidence quality were all consistent with the application performed by the present study. However, there is a considerable difference regarding the reasons for the decrease or increase in the reliability of outcomes, especially for assessment of ‘risk of bias’ and ‘indirectness’ (main drivers of decreasing certainty of evidence). This demonstrates how the main factors that reduce the degree of confidence in the outcomes refer to the main basics of methodological conduct and design of the research question. Finally, all remaining SRs did not employ any tools to assess the certainty level or the risk of bias of their outcomes with GRADE or risk of bias approaches, respectively.

Up-downgrades items

All nineteen outcomes assessed experienced some form of downgrading of the certainty level of the evidence, amounting to a total of sixty downgrades. The main reasons found were indirect evidence (26.6%), inconsistency (23.3%), and imprecision (23.3%), with the same frequency of evidence decreasing for the latter two. In line with this, risk of bias was also responsible for a significant cause of decrease in the reliability of outcomes (21.6%), given the substantial number of observational studies included in the SRs, and followed lastly by Publication Bias (5%). Eight increases in certainty occurred solely due to the Large Effect Size of the evidence, with no observed reasons for an increase based on Dose-Response Gradient or Residual Confounding. Downgrades were more frequent in SRs of NRSs (71.6%), while a smaller portion focused on those of RCTs (28.3%). On the other hand, increases contrast in terms of prevalence, found in a higher proportion in RCT-based SRs (62.5%) versus observational NRSs ones (37.5%).

Discussion

This study aimed to assess the quality landscape of SRs conducted in LMICs concerning chronic NCDs and their most significant modifiable risk factors, using scientifically supported assessment instruments [24, 27, 31, 34]. Firstly, the comparison between the results of PRISMA 2009 and PRISMA 2020 was not possible because the quantity and nature of the items are different, making a comparison unfeasible. However, articles published after the launch of the PRISMA 2020 update [41,42,43] had a higher overall score than articles published before 2020. This suggests that the new update, being more detailed, is capable of guiding a more comprehensive reporting of SRs and meta-Analyses. Comparing the results of the articles that did not mention the use of the PRISMA checklist [35, 37, 43] with the rest of the articles that mentioned the use of the tool, it becomes clear: the sum of items considered ‘not informed’ is higher among the articles that did not follow the PRISMA (Figs. 2 and 3). This highlights the importance of using the checklist when writing the systematic review.

The application of the TIDieR checklist reveals how many essential characteristics for a comprehensive description of interventions are met, including the description of the name, justification, implementation procedures and application locations. However, there is a significant drop in references to the tool’s items, especially in relation to “costs involved” and “time to deliver the intervention”. One possible reason for this finding may be the indirect understanding by SRs’ writers that the use of the intervention is delimited as the total study duration, from the beginning to the end of obtaining results, for example. Despite that, this way of thinking distorts the understanding of the exact moment it was employed, thereby hindering the acquisition of details regarding relevant issues, such as whether the total time was adequate for obtaining the number of outcomes used, if there were interruptions and new attempts to apply the intervention, the duration and quantity of sessions for its delivery, understanding if the time was within the necessary timeframe for the total study conduct, and among other considerations. Additionally, there is a low characterization of the intervention providers and the materials used to implement it (5/9–56% correspondence), according to the TIDieR evaluation. This finding may indicate a turning point in the analysis of interventional studies, especially those with sponsors or stakeholders relevant to their conduct, which may not be fully reported to conceal the decisive role of external forces in the study’s execution.

Nevertheless, adherence and fidelity to interventions were scarce in the studies analyzed (2/9–23% correspondence), indicating slow progress in investigating the incorporation of interventions by participants. Items 11 and 12 from TIDieR tool, for example, should be among the well-reported in this type of interventional study, as the results depend directly on the intensity of internalization and consequent practice of the assigned interventions. Consequently, the reliability level of the outcomes obtained in participant groups subjected to any intervention may be partially or entirely affected by the fidelity of its implementation. Finally, the non-identification of modifications and intervention personalisation (0/9 correspondence) reveals an ambiguous scenario in SRs, where there is a commitment to completing planned interventions without significant changes, but at the same time, the lack of adaptations ignores the peculiarities of the participants. In a setting of multiple modalities and quality levels of health surveillance systems, as is the case with low- to middle-income populations, this may lead to participant dropout or a lack of complete especially in health systems with low quality to support participants adherence, thus compromising items 11 and 12 as well.

The prevalence of systematically low and critically low methodological quality SRs on chronic NCDs theme and their risk factors in LMICs was 88.89% of the articles assessed in this meta-research. This reality raises concerns about the questionable quality of available evidence from these countries due to shortcomings in methodological detailing, compromising the reliability and reproducibility of these reviews based on the AMSTAR-2 tool. The majority of articles failed to provide detailed and justified descriptions of how the SRs were conducted. The absence of a prior protocol and justifications for deviations (item 2) directly impacts the reproducibility and credibility of the study, with 33% of the studies not meeting this domain. The inadequate detailing of the search strategy (item 4) in more than half of the studies also affect reproducibility, while the lack of a list of excluded studies with justification (item 7) increases the risk of selection bias, reducing transparency. In this meta-research, 88% of the studies failed in methodological detailing. The Cochrane Handbook mentions that having a written protocol, specifying the scope and methods to be used in the review, assists in planning and reduces the risk of bias in the review [12]. The articles reported few justifications for including a particular study design, in addition to not presenting excluded studies and justifying their exclusion. Consequently, there are several information gaps and a lack of accountability regarding decisions in terms of methodological steps.

AMSTAR-2 is a tool more objective and rigorous by eliminating the option to categorize as “not applicable” or “cannot be answered”. Consequently, the absence of a list of excluded studies, present in eight out of nine articles, contributed to the low methodological quality classification. The question of the relevance of item 7 as a critical domain is raised, as, although mentioned in the Cochrane manual, it is not mandatory in the PROSPERO or PRISMA guidelines. Moreover, for the record, 44% of the SRs with meta-analyses did not follow all the principles recommended for their construction, which did not necessarily impact the landscape of methodological quality classification, due to the fact that the construction or not of an meta-analyse takes into account multiple factors.

The GRADE approach, to assessing the quality of evidence, showed low quality in the outcomes of the selected studies, with 17 classified as very low [36,37,38, 42, 43] and 2 as moderate [36]. Factors such as indirect evidence, inconsistency and imprecision most compromised their quality. This reveals that associations between chronic diseases and modifiable risk factors are often presented in SRs with a low level of evidence, despite being studies at the top of the scientific evidence, with the objective of generalizing results and guiding several professionals around the world for their clinical practice [14, 15]. This scenario reveals significant issues, indicating a lack of approaches to assess the level of evidence for outcomes in people from LMICs. Besides the scarcity of studies addressing the theme for their populations, the association between chronic NCDs and behavioral risk factors is not well understood among LMICs populations. In this regard, the application of the tool has shown us that such an association can be better recognized, specifically demonstrating how these pieces of evidence exhibit significant deficiencies in quality. Therefore, according to the results obtained in our study, it should be acknowledged that LMICs suffer from a low quantity and even low quality of robust health evidence for an established association between chronic NCDs and modifiable risk factors.

Furthermore, authors of SRs that evaluated the association between chronic NCDs and behavioral risk factors in populations from LMIC rarely applied the GRADE tool, as demonstrated by the presence of only two studies among those selected. The remaining articles, by not applying GRADE, indicated a summary approach in the form of qualitative synthesis made from quantitative results to infer the overall quality of the evidence [36, 42]. This justifies the inability to use the tool since GRADE was not designed for the body of evidence from qualitative SRs. The low number of assessments of the body of evidence, on the other hand, reinforces the low level of GRADE application, even in a sample universe of nine assessments and after 23 years of its creation. Such assessments, in fact, were present in the highest-ranked articles by the PRISMA and AMSTAR-2 checklists in our study, suggesting the possibility that the tool is more notably recognized and likely to be applied in studies with higher reporting and methodological robustness.

The comparison between SRs conducted in High Income Countries (HICs) and LMICs reveals differences in adherence to these guidelines. Studies in HICs, for example, generally show greater consistency in the application of quality assessment tools, as demonstrated by Shea et al. [25], who demonstrated how SRs conducted in HICs tend to more rigorously meet the criteria established by AMSTAR-2, particularly in terms of critical assessment of biases and the justification for inclusion and exclusion of primary studies; and Mustafa et al. [30], who observed a higher frequency of proper application of the GRADE tool in SRs from HICs and how LMICs are hampered by a lack of digital resources, that directly affect researchers’ ability to fully apply methodological guidelines. This limitation can result in studies with lower methodological rigor and, consequently, inferior quality of the evidence presented, already seen in several LMICs studies failing to fully adhere to established guidelines [44,45,46,47,48].

Strengths and limitations

This study is the first at the forefront for an analysis in all aspects of understanding an SR with/without meta-analysis using PRISMA, AMSTAR-2, TIDieR, and GRADE tools to comprehend their conception process, writing quality, interpretation methods and practice applicability, in order to mitigate possible selection, methodological and publication biases. In addition, we also did look at the year that each trial was published, which allowed us to evaluate the trend in studies being published to determine if the quality of the literature is improving over the time of publication from the included studies. Therefore, our study can be useful in the systematic and comprehensive methodological understanding of the most important and recent SRs that assess the chronic pathological effects of major behavioral health factors in populations from LMICs, an area that has been gaining prominence on the world stage for decades [49].

However, our study also has limitations that are worth describing. The first one relates to the low body of evidence analyzed, as a result of the lower number of registers about NCDs and lifestyle in LMICs, which affects the degree of generalization of the existing evidence. Furthermore, there were several doubts and ambiguities in identifying the characteristics of the assessed risk factors. The process required the involvement of several data extractors due to its time-consuming nature, which was further complicated by the way the details were reported or, in many cases, by the lack of such details, thus becoming a possible source of information and reporting bias in our study. However, it was possible to circumvent this possible bias through a thorough search of epidemiological information from SR studies in their original databases and their pre-registrations, as recommended by Cochrane Risk of Bias Tool and GRADE approach [12, 21, 28]. In addition, there was a small number of SRs founded in health literacy about this thematic area, despite a sensitive and specific search strategy in order to mitigate a possible selection bias. First, it raises awareness of the low production of studies focused on this profile of vulnerable populations from LMICs, in contrast to the prolific health literature with populations from HICs. Second, we also have to cite the existence of so many other scientific design sources of the thematic area not included in this paper due to their limitations in representing traditional streams of robust evidence, such as case–control, cohort studies and non-systematic reviews specific to LMICs. Therefore, this may have limited the representativeness of the results, since the exclusion of studies with higher quality evidence than the SRs evaluated may have underestimated the effect of the risk factors evaluated on the emergence of NCDs.

Implications for clinicians and approaches for future research

The implications of this study are profound and suggest the need for targeted interventions to improve the methodological quality of SRs in LMICs. First, the few studies found in the current literature that deal with the chronic NCDs and lifestyle risk factors show the need for more studies in the area to understand this association in LMICs, which may occur due to the lack of visibility of developing nations within the global scientific scenario focused on HIC, demonstrated by its high annual article publication rates [50, 51]. An effective approach to this situation could involve promoting international collaborations with researchers from HICs and LMICs. A team work between nations can facilitate the sharing of knowledge, techniques and resources, raising the standard of research in LMICs and ensuring that results are globally relevant and applicable. It takes into account some LMICs financial matters, such as the few investments in the ​​public health research area. This also leads us to recommend significant participation of these countries with non-commercial scientific institutions to change this paradigm, due to scarce funding to their healthcare systems. Another possible advice to solve this situation is to recommend that these countries improve their health system infrastructure of collection and storage of data.

Furthermore, the certainty of evidence of the included studies in this meta-research raise significant concerns regarding the applicability of the current evidence from NCDs to LMICs, due to the paucity of contextual trial data and compliance with data construction precepts long envisioned by established tools for conducting SR, such as GRADE tool. Otherwise, encouraging investors to create programs that provide tools and necessary training to researchers in LMICs to conduct high-quality SRs is an effective solution for this problem. Accordingly, researchers can adequately apply tools like PRISMA, AMSTAR-2, TIDieR, and GRADE, which could significantly enhance the quality of evidence emerging from these countries. This thematic area of ​​chronic diseases related to lifestyle requires evidence that sustains high quality when evaluated in low-income populations, a fact that has long been demonstrated in individuals with HICs [52, 53].

Conclusion

Our results revealed that the writing of recent scientific articles on the association investigated in this study does not meet most of the criteria provided by the recent version of the PRISMA 2020. The checklist was the most reported and used tool in the studies, being the writing of SRs its best-rated aspect. Moreover, the TIDieR checklist presented how the applied interventions are very well related, meeting almost all of the criteria for their characterization. Nevertheless, TIDieR was not cited as a guided tool for any of the included studies. Regarding the methodological conduct aspect, the AMSTAR-2 tool revealed mixed results regarding the SRs evaluated. Among them, the evaluation of moderate quality of conduct, but mainly low and critically low methodological conducts. The certainty of evidence has a predominantly very low quality too, classified by the GRADE approach, also not widely utilized in the landscape of SRs addressed in our study. Therefore, we observed a high quality in the SRs writing and interventions, despite the low quality of their health outcomes and methodological conduct. Encouraging adherence to these approaches in the field of SRs addressing associations between chronic NCDs and behavioral factors in individuals from LMICs needs to be enhanced and reinforced to change this scenario.