Hello, I'm Nury Agustí, a gynecologic oncologist currently performing a postdoctoral fellowship at MD Anderson Cancer Center. In this lecture, we are going to talk about the systematic review and meta-analysis. So, a systematic review is a comprehensive and structured approach to summarizing, evaluating, and synthesizing existing research evidence on a specific research question or topic. It involves a meticulous and transparent process that aims to minimize bias through the use of pre-specified research questions and methods that are documented in protocols. Reviews play a crucial role in summarizing and evaluating existing literature. However, each review type has a different purpose. For example, narrative reviews are characterized by their less structured approach, often relying on the author's subjective interpretation of the literature, so they are more prone to bias and less reliable compared to systematic reviews. And, on the other side, the meta-analysis takes systematic reviews a step further by performing statistical analysis on data extracted from individual studies included in this review. Now, let's answer why systematic review and meta-analysis are so important. Basically, they play key roles in guiding our choices and shaping the future of medical research. For example, systematic reviews might guide doctors in choosing treatments. They shape evidence-based guidelines, empower patients' physicians' discussions, or, for example, identify research priorities and help allocate research and resources effectively, ensuring cost-effective approaches. So, to kick-start a systematic review, the foundational step is to establish your review team. Just like a mission needs a skill crew, your systematic review needs expertise. Then, consider the Cochrane Handbook for Systematic Reviews of Intervention, which offers guidance for rigorous evidence-based reviews. So here is a 10-step guide on how to conduct a systematic review and meta-analysis, and we will review each step one by one. The first and most crucial step in preparing a systematic review is defining its focus. To achieve this, we can follow the final criteria. So, ensure the research question is feasible and can be answered with available data, avoiding empty reviews. Make sure the question is interesting and meaningful for healthcare decision-making. Determine if the research question is novel and adds new knowledge to existing literature. Verify the research question, respect the rights and well-being of participants, and also consider whether the findings would be relevant to enforce decisions effectively. In step two, you should frame your research question, since the type of question you choose to address in your systematic review can significantly impact the design, methodology, and focus of your review. Here are some examples. Once we have the scope of the review, and we know it's worth pursuing, we have to formulate the frame of the research question using the acronym PCOP. This acronym stands for Population, Intervention, Comparison, Outcome, and Time. As we delve into the P of Population, let's remember that setting clear and non-ambiguous eligibility criteria is essential for conducting a systematic review. These criteria should be designed to strike a delicate balance, broad enough to encompass the diversity of studies, yet narrow enough to provide meaningful results when we consider all the studies together. Regarding the other components of the acronym, let's briefly go over them. For the I for the Intervention defines the specific treatment or exposure we are focusing on. The C for Comparison assesses whether there is another treatment or exposure for comparison. The O for Outcome defines the expected outcomes or results that we will assess. And the T for Time establishes the time frame within which we measure these outcomes. So, in summary, the PCOP question serves as a predefined format for structuring review questions. Let's move to the next step regarding the search strategy. In the search for relevant studies, collaboration is crucial. Don't go it alone. Tap into the expertise of medical librarians and information specialists. Universities and hospitals often have these professionals. When conducting, when structuring your search strategy, incorporate PCOP concepts and eligibility criteria into the search strategy to guide the selection of the relevant words. However, it's important to find a balance. You don't need to search every aspect of your clinical question. Finally, your search strategy typically involves three sets of terms. Keywords related to the health condition of interest. For example, cervical cancer. Terms associated with the intervention and the evaluation. For example, surgery versus radiotherapy. And lastly, keywords that identify the type of study design. For example, randomized clinical trials and observational studies. So, now that we have gathered our terms, the next step is to combine them effectively. We can use connectors like and, or, or not to refine our search. Also, when constructing our search strategy, we can combine controlled vocabulary, also known as mesh terms and text words, such as synonyms and related terms. Additionally, we can search filters to refine our question. Here is an example of a search strategy executed in the EMBASE database. So, for the following step, we strongly recommend registering your systematic review protocol on Prospero. It is an online platform dedicated to this purpose. And why is this important? It allows researchers to transparently document their plans for conducting systematic reviews before they begin the review process. This registration helps prevent duplications of effort, enhances research transparency, and promotes the rigor and quality of systematic reviews. To note, you should include this protocol and the PICOT questions. Search strategy with the sources used and the methodology you will use, including all the following steps that we are now going to review. So, before moving to the next step, it is important to know that Prisma Flowchart, what Prisma Flowchart is. Review authors should use the Prisma Flowchart to document the selection and screening of studies at each stage. This diagram guides us our step-by-step scan, so it's really important. Now we are moving on to step five. Once we have designed our search strategy, it's time to put it into action. We should execute this strategy in at least two separate databases. And why two? Well, each database might hold a different set of articles. So, by searching multiple databases, we ensure covering and reducing the risk of missing relevant studies. Speaking of databases, three stand out as crucial sources when searching for reports of trials. Those are the MEDLINE, EMBASE, and CENTER. For broader coverage and a wider range of disciplines, researchers can also explore other databases like Web of Science and Scopes. And expanding beyond just MEDLINE, usually through PubMed, is vital, since relying solely and only on MEDLINE could result in an unrepresentative set of reports or studies. And beyond these well-known databases, there are local and subject-specific ones that can help to explore appropriate national, regional, and subject-specific bibliographic databases to ensure comprehensive coverage. And finally, let's not forget OpenGrade, a valuable resource for great literature that includes theses, conference papers, government publications, and much more. So, after executing our search strategies across at least two databases, we find ourselves with results from different sources. So, at this stage, it is crucial to identify and remove duplicate citations that appear in the multiple databases. To streamline this process and ensure precision, we highly recommend employing reference management software. One of the most well-known ones is Covidence. It is a systematic review management tool that streamlines the process of conducting systematic reviews. It enables collaboration among multiple reviewers and enhances transparency through the review process. For the next step, we move to the screening of titles, abstracts, and full text. In the initial phase, we should involve at least two independent reviewers to assess the studies. Additionally, we should pre-establish a conflict resolution plan and consider third-party arbitration if needed. It is also important to maintain a list of excluded studies. This list should cover all studies that might initially appear to meet the eligibility criteria but, upon closer examination, do not qualify. Finally, as conducting systematic reviews often involves sifting through thousands of records, to address this challenge, various software tools and packages other than Covidence can be also used and is highly recommended. Now, as we transition to the next phase of our systematic review process, let's talk about the data extraction process. What exactly do we mean by data? In the context of a systematic review, data includes any and all information related to the study. This includes methodological details, participant demographics, study settings, contextual information, intervention-specific outcomes, and more. The data collected during this phase must adequately describe the included studies, support the creation of tables and figures, assist in evaluating the risk of biash, and ultimately enable the synthesis and meta-analysis of findings. However, it's important to note that centers, authors might find missing study details such as incomplete outcomes or numerical results. In such cases or situations, reaching out to the original investigators or authors is strongly recommended to fill these information gaps. So, moving forward, let's analyze the risk of biash in our InClarity studies. Biash, in this context, refers to systematic errors that deviate from the true study results, potentially leading to underestimation or overestimation of the intervention effect. It's essential to know that biash should not be confused with imprecision or external validity. To help us in this task, we use structured risk of biash tools. They have different sections, each looking at the specific aspects of the study's quality. And here we can see a brief overview of some common risk of biash tools that tailor different types of studies, such as ROP2, which is designed to assess the risk of biash in randomized control trials, Rawlings-1 or Newcastle-Ottawa scale, which are used for non-randomized or observational studies, and QuadS2 that is used for diagnostic accuracy reviews. And for the next step, we move to meta-analysis section. It offers a quantitative synthesis of data from multiple studies on a particular research question. Let's go through its primary purposes. Pooling effect sizes. Meta-analysis combines effect sizes from individual studies to provide an overall estimate of the treatment effect or relationship between variables, along with a confidential interval. Increasing statistical power. It is particularly valuable when dealing with studies with limited sample sizes. Aggregating data from multiple studies boosts statistical power. Detecting patterns and heterogeneity. We can identify data patterns and assess variability across studies, ensuring result consistency. Finally, it also allows to explore factors that may influence study outcomes through subgroup analysis, identifying potential modifiers of the treatment. As we progress through the systematic review process, the question of when to perform a meta-analysis is a crucial one. Let's explore the factors that guide our decision in this regard. When to do a meta-analysis? First and foremost, a meta-analysis becomes a feasible option when we have multiple independent studies that estimate the same treatment effect or association for our research questions. Furthermore, it is important that the treatment effect is consistent across these studies and has been measured and reported in similar ways. And lastly, we can also do a meta-analysis when the differences in the study characteristics are unlikely to substantially affect the treatment effects. And when not to do a meta-analysis? This is also important. There are situations where we should exercise caution and avoid conducting a meta-analysis. Firstly, we should avoid it if the studies included are of low quality. Keep in mind that a meta-analysis is only as reduced as the studies included. Combining low-quality studies can lead to unreliable results. Additionally, if we come across biased studies, it's best not to combine them in a meta-analysis, neither. Moreover, if the studies in our review are too diverse, a meta-analysis may not provide us with specific meaningful insights. And lastly, we must ensure that the studies we include collectively address a similar overreaching question, even if it's a broader one. This ensures that our comparisons are relevant. I would like to briefly mention the RITE system. It is a systematic framework for forming clinical practice recommendations. It emphasizes transparency and balances assessments, particularly when rating evidence quality for different outcomes. In the final stage of our systematic review process, we transition from the data synthesis to the crucial task of writing the report. This is where we present our findings in a clear and organized manner. To ensure that our reports meet the highest standard of transparency and quality, we can use standardized styles and checklists such as Prisma and AMPster. Here are some key points to keep in mind. First, organize the report logically with clear headings to guide readers to the content. Also, discuss your search strategy, eligibility criteria, and the study selection process, ideally using the Prisma flowchart to provide a visual overview. Present your results effectively using tables, figures, and narrative summaries to convey the key findings. Also, engage in a discussion that includes honest assessments of limitation and potential biases. And most importantly, are there to establish reporting guidelines such as Prisma. So in this slide, we can find a section of the Prisma checklist. And Prisma offers a structured framework designed explicitly for systematic views and meta-analysis. It covers all the aspects of our report. Adhering to this checklist is strongly encouraged by journals, publishers, and institutions. And finally, I would also like to briefly mention the AMPster tool. AMPster is used for assessing the quality of systematic reviews. It includes a checklist of 16 items that reviewers use to evaluate various aspects of a review's methodology. So which are the take-home messages? First, teamwork is essential for systematic reviews involving clinicians, epidemiologists, and methodologists. Use available resources to simplify your work. Prioritize protocol development and registration for guidance. Consider the need for meta-analysis in each review. And ensure a minimum quality report using Prisma and AMPster tools. Thank you very much for your kind attention.

systematic reviews

meta-analysis

medical research

evidence-based guidelines

research priorities