Effects of maternal influenza vaccination on adverse birth outcomes: A systematic review and Bayesian meta-analysis
Effects of maternal influenza vaccination on adverse birth outcomes: A systematic review and Bayesian meta-analysis
Sohyun Jeong 0 1 2
Eun Jin Jang 1 2
Junwoo JoID 1 2
Sunmee JangID 1 2
0 Hinda and Arthur Marcus Institute for Aging Research, Hebrew SeniorLife and Harvard Medical School , Boston , Massachusetts, United States of America, 2 School of Pharmacy, Sungkyunkwan University , Jangan-gu, Suwon, Gyeonggi-do, Korea , 3 Department of Information Statistics, Andong National University , Gyeongsangbuk-do, Korea , 4 Department of Statistics, Kyungpook National University , Bukgu, Daegu , Korea , 5 College of Pharmacy and Gachon Institute of Pharmaceutical Sciences, Gachon University , Incheon , Korea
1 Editor: Lamberto Manzoli, Universita degli Studi di Ferrara , ITALY
2 The Cochrane Library, PubMed, EMBASE, Web of Science , and Scopus were searched
Data Availability Statement: All relevant data are
within the manuscript and its Supporting
Funding: This work was supported by a grant
(17171MFDS278) to S. Jang from the Korea
Ministry of Food and Drug Safety. However, the
funder had no role in study design, data collection
and analysis, decision to publish, or preparation of
Competing interests: The authors have declared
that no competing interests exist.
Study eligibility criteria
This analysis included randomized placebo-controlled studies, cohort studies, and
casecontrol studies, in which inactivated influenza vaccination was given during pregnancy and
fetal adverse birth outcomes were assessed.
Participants & intervention
Women who received inactivated influenza vaccine during pregnancy and their offspring.
Study appraisal and synthesis
Two independent reviewers and a third reviewer collaborated in study selection and data
extraction. A Bayesian 3-level random-effects model was utilized to assess the impact of
maternal influenza vaccination on birth outcomes, which were presented as odds ratios
(ORs) with 95% credible interval (CrIs). Bayesian outcome probabilities (P) of an OR<1
were calculated, and values of at least 90% (0.9) were deemed to indicate a significant
Among the 6,249 identified publications, 48 studies were eligible for the meta-analysis,
including 2 randomized controlled trials, 41 cohort studies, and 5 case-control studies. The
risk of none of the following adverse birth outcomes decreased significantly: preterm birth
(OR = 0.945, 95% CrI: 0.736?1.345, P = 73.3%), low birth weight (OR = 0.928, 95% CrI:
0.432?2.112, P = 76.7%), small for gestational age (OR = 0.971, 95% CrI: 0.249?4.217,P =
63.3%), congenital malformation (OR = 1.026, 95% CrI: 0.687?1.600, P = 38.0%), and fetal
death (OR = 0.942, 95% CrI: 0.560?1.954, P = 61.6%). Summary estimates including only
cohort studies showed significantly decreased risks for preterm birth, small for gestational
age and fetal death. However, after adjusting for season at the time of vaccination and
countries? income level, only fetal death remained significant.
This Bayesian meta-analysis did not find a protective effect of maternal influenza
vaccination against adverse birth outcomes, as reported in previous studies. In fact, our results
showed evidence of null associations between maternal influenza vaccination and adverse
Pregnant women are deemed vulnerable to severe complications from influenza infection due
to changes in the immune system and heart and lung function during pregnancy [
infection can also affect fetal development, leading to adverse birth outcomes [
]. Based on
the evidence of the landmark Mother?s Gift trial in Bangladesh , the World Health
Organization (WHO) Strategic Advisory Group for Experts on Immunization recommended in 2012
that pregnant women be considered a priority group for influenza vaccination [
highincome countries have adopted and incorporated the WHO recommendations in their
maternal-child health strategies [
]. Numerous studies have shown inactivated influenza vaccination
in pregnant women to be effective at protecting women and their offspring via transplacental
antibody transfer [
]. The two-fold benefit of inactivated influenza vaccine in preventing
laboratory-confirmed influenza infection, both in pregnant women and infants, is convincing
], especially as it has been proven in recent, large-scale, phase 4 randomized controlled
trials (RCTs) conducted in Mali [
], Nepal [
], and South Africa [
However, the influenza vaccine uptake rate varies widely, ranging between 4% and 93% [
], and concerns about the risk to the fetus are a major reason why women sometimes do not
receive vaccination during pregnancy, especially in many low/middle-income countries where
maternal influenza vaccination has not been incorporated into routine immunization
]. To present solutions for this issue, many phase 4, post-marketing surveillance
or observational studies have been conducted to evaluate the fetal outcomes of preterm birth
(PTB), small for gestational age (SGA), low birth weight (LBW), congenital abnormalities and
fetal death. Many of those studies have suggested that influenza vaccination during pregnancy
2 / 17
was not associated with adverse birth outcomes [
], but some reported negative [
] results for each of the above birth outcomes. A few systematic reviews and
meta-analyses were conducted, with similarly inconsistent results; maternal influenza
vaccination reduced the incidence of PTB [
], SGA, LBW , and stillbirth (or fetal death) [
], but no association was found with congenital malformations [49, 50], fetal death, or
spontaneous abortion . However, Vazquez-Benitez et al. raised the issue that biases in
conducting observational studies of maternal influenza vaccination may lead to false positives .
Donzelli  also explicitly stated that positive results from observational studies might be
due to confounding by indication and healthy-vaccine bias. Furthermore, a case-control study
funded by the US Centers for Disease Control and Prevention (CDC) reported that women
who were given maternal flu vaccination containing the pandemic H1N1 (H1N1pdm09)
component in the prior season had an increased risk of spontaneous abortion within 28 days after
the next seasonal vaccination . Research is continuing into this issue, and the CDC has not
confirmed that the maternal flu vaccine causes spontaneous abortion .
These results indicate that the evidence for adverse birth outcomes is inconsistent and
lacking methodological robustness, as it is largely dependent on observational studies. Critiques
regarding the limitations of ecological studies?including statistical imprecision and clinical
and methodological heterogeneity?that make them prone to substantial bias have argued that
more sound research is needed, in addition to previously published systematic reviews and
meta-analyses . Therefore, given the large volume of observational studies and the paucity
of RCTs available, we aimed to overcome the heterogeneity of studies by conducting a
Bayesian hierarchical meta-analysis, and to present new evidence regarding the association between
maternal influenza vaccination and adverse fetal birth outcomes.
A systematic literature review was conducted conforming to the Preferred Reporting Items
for Systematic Reviews and Meta-Analyses (PRISMA) recommendations . The Cochrane
Library, PubMed, EMBASE, Web of Science, and Scopus were searched using the following
keywords: ?pregnan ?, ?influenza?, ?vaccine,? and ?immunization,? filtered with ?human?
without limitations on the date up to May 2019. A manual review was performed to search for
other relevant articles in the references of selected articles and WHO publications. The
inclusion criteria were 1) human studies; 2) inactivated influenza vaccination given in any period
of pregnancy; 3) reporting the target outcomes of PTB, LBW, SGA, fetal death, and congenital
malformations; 4) RCTs, prospective or retrospective cohort studies, case-control studies; 5)
placebo-controlled studies, and 6) the availability of full-text published articles in the English
language. The exclusion criteria were 1) single-arm studies; 2) cross-sectional studies; 3) active
controlled studies; 4) animal studies; 5) studies not on the target outcome, target vaccine, or
target population; 6) reviews, abstracts for posters, editorials, commentaries, or letters; 7)
studies not written in English; and 8) studies with no available full text.
Two independent reviewers (MS & JL) searched for and selected the final studies through
consensus. An additional third reviewer (SJ) also participated in the process and reached the
same consensus on the final selection.
We retrieved relevant information from the final selected studies: publication year, name of
the first author, total number of participants, vaccine type (trivalent, monovalent, or
MF59adjuvant), the period when vaccines were given (pandemic or seasonal), country of the study,
3 / 17
income level of the country where the study was conducted, study design, and the effect sizes
and confidence intervals reported in the studies.
Assessment of risk of bias and publication bias
The quality of the included studies was independently assessed by 2 researchers (JL & MS).
RCTs were graded by the Risk of Bias tool developed by the Cochrane Library, whereas cohort
and case-control studies were assessed by the Newcastle-Ottawa Scale (NOS). Funnel plot
visualization and the Egger regression method were used, since they are widely accepted tools for
assessing publication bias.
Statistical analysis: Bayesian meta-analysis
A Bayesian meta-analysis was performed to assess the association between adverse birth
outcomes and influenza vaccination during pregnancy. The Bayesian approach is superior to
traditional meta-analyses, especially for rare adverse events . In the Bayesian method, the
posterior probability distribution is obtained by combining a prior distribution with a
likelihood distribution from the observed data, and the quantity of interest can be directly
interpreted as the probability . Since this study included cohort studies, case-control studies,
and RCTs, to consider the heterogeneity among study design, a hierarchical Bayesian 3-level
random-effects model was used . A hierarchical Bayesian random-effects model is useful
when incorporating the imprecision of the variance estimates of treatment effects within
studies, the imprecision in the estimated between-study variance estimate, and explicitly modeling
binary outcome data . The odds ratios (ORs) and 95% confidence intervals (CIs) or sample
size and the count of the outcome were extracted from each study. When an adjusted OR was
presented in the cohort study or case-control study, it was first extracted and used as the effect
size. When the effect size was presented as a hazard ratio (HR) or relative risk (RR), it was
considered as an OR. Since the underlying assumption of a generally low event risk (<20%) can
be made for adverse birth events, and the HRs and RRs are very close to 1, it can be logically
concluded that HRs, RRs, and ORs will be relatively close to each other [61?63]. ORs were
calculated when the sample size and the number of outcomes were present in a study, and ORs
were calculated after a continuity correction when there were no adverse birth outcomes in
the vaccinated pregnant women or the unvaccinated pregnant women. The ORs calculated for
each study were converted to log (OR) values, which were used as the effect size in the Bayesian
We presented the posterior median and 95% credible interval (CrI) of the overall ORs and
ORs for each study design (RCTs, cohort studies, case-control studies). The main goal of this
study was to investigate whether the risk of targeted birth outcomes was lower in the
vaccinated group than in the unvaccinated group. To verify this hypothesis, the posterior
probability (P) of the OR being less than 1, which can also be denoted as P(OR<1), was estimated.
Since a probability of 50% can be interpreted as a null effect, whereas a probability of at least
90% indicates a significantly beneficial effect of maternal vaccination in decreasing adverse
birth outcomes , a probability of over 90% was considered significant in this study. To
explore the effects of covariates such as the vaccination period (seasonal or pandemic) and the
income level of the country where the study was conducted, Bayesian meta-regression was
performed. If the 95% CrI of regression coefficient did not contain 0, the covariates were deemed
The non-informative prior distribution was considered as the prior distribution, and the
convergence of the Markov-chain Monte Carlo (MCMC) algorithm was assessed using the
trace plot, autocorrelation plot, and the Gelman-Rubin statistic. In total, 110,000 iterations
4 / 17
were performed in each of three chains. The first 10,000 iterations were burn-in to eliminate
the initial value effect, and each 10th value among the remaining 100,000 values was used to
estimate the posterior distribution. When the posterior median of the standard deviation
among studies (heterogeneity, ?) was equal to or lower than 1, the heterogeneity was deemed
non-significant [66, 67]. The funnel plot method and the Egger test were used to explore
publication bias in the included studies. All models were fitted in WinBUGs 1.4.3 (Medical
Research Council Biostatistics Unit, Cambridge, UK) using MCMC algorithms.
PRISMA flow for study selection
Among the 6,249 identified publications, 48 studies were eligible for the meta-analysis.
Duplicate articles (n = 2,901) were excluded in the first step, and 3,223 articles were removed as
inappropriate by title and abstract review. One hundred and twenty-five articles were eligible
for full-text review. After excluding studies that did not investigate our target population
(pregnant women or their infants) (n = 11) or the targeted birth outcomes (n = 47); provided
inadequate information (n = 1); or were single-group studies (n = 16), an active controlled
study (n = 1), or a cross-sectional study (n = 1), 48 studies were finally selected for the
metaanalysis (Fig 1).
Fig 1. PRISMA flow diagram of study selection for the meta-analysis.
5 / 17
Characteristics of included studies
Two RCTs, 5 case-control studies, and 41 cohort studies were included. According to the
world income data of the period when the studies were conducted, 6 of them were from lower/
middle-income countries  including 2 large RCTs (Nepal, South Africa), China, Nicaragua,
Taiwan and Laos. Forty-one studies were from high-income countries , mainly located in
North America and Western Europe (S1 Table).
Twenty-five studies reported birth outcomes associated with monovalent pandemic
vaccination, 10 studies reported outcomes from the trivalent inactivated influenza vaccine, and the
rest either did not indicate the vaccine type or indicated both types. A total of 2,387,107
pregnant women or their offspring were included in the 48 studies. The smallest study had 226
subjects and the largest had 425,944 subjects. The source of observational studies was generally
pregnancy or birth or prenatal registries, vaccine registries, vaccine safety data linkage, hospital
records, and surveillance data.
Quality assessment of included studies: Risk of bias and publication bias
The quality of the included studies was independently assessed by 2 researchers. RCTs were
graded by the Risk of Bias tool developed by the Cochrane Library  (S1 Fig), whereas the
NOS  was used to check the quality score of cohort studies and case-control studies. As the
selected primary literature had a low risk of bias in the domain of baseline outcome measures
and characteristics, the baseline characteristics between the two groups were similar. The
reporting of results likewise had little risk. However, the risks regarding blinding, allocation
concealment, and contamination were high due to the nature of observational studies.
Additionally, we stratified the NOS quality score of the included observational studies according to
the Agency for Healthcare Research and Quality (AHRQ) conversion standards [
46 observational studies, 37 were of good quality and 9 studies were of fair quality (S2 Table).
As widely accepted tools for publication bias, funnel plot visualization and the Egger
regression method were used. The Egger regression method did not detect publication bias for PTB
(p:0.709), LBW (p:0.309), SGA(p:0.203), congenital malformations(p:0.972) or fetal death
(p:0.670). Funnel plots of all the analyses suggested that most studies had a large sample size,
clustering in the upper part of the funnel plot. The presence of no studies in the lower-left part
of the funnel plot for LBW suggests that small studies finding an increased risk of LBW may
have remained unpublished (S2 Fig).
Association between maternal influenza vaccination and adverse birth
Each birth outcome was assessed by the Bayesian random-effects model and the risk of bias
across studies (heterogeneity) was assessed by the posterior median of standard deviation
among studies (SD, ?). All the studies included and evaluated in each adverse outcome showed
low heterogeneity (SD close to 0) (Table 1). The effects of the vaccination period (seasonal or
pandemic) and the income level of the country where the study was conducted on each birth
outcome were evaluated by meta-regression analyses. Only covariate of high income country
in PTB analysis (beta: -0.141, 95% CrI: -0.276-(-0.004)) showed significantly protective effect
and none of the other covariates showed statistically significant effects, as the 95% CrIs for
their coefficients (?) included 0 (S3 Table). However, after adjusting for the effects of
covariates in cohort summary estimates, the probability (P) of protection against PTB, LBW and
SGA was significantly reduced and only protective effect against fetal death remained
significant (Table 1).
6 / 17
CrI: Credible Interval,
SD (?): Standard deviation between studies (estimation of heterogeneity between studies)
Bayesian meta-analysis of PTB. Thirty-two studies, 2 RCTs, 28 cohort studies (1 cohort
] reported results from 2 different groups, so 29 data sets were included), and 2
casecontrol studies were included in the PTB analysis (? = 0.094,low heterogeneity). The overall
PTB risk was not significantly decreased in the vaccinated group (OR = 0.945, 95% CrI: 0.736?
1.345, P = 73.3%). The risk of PTB in RCTs and case-control studies was not significantly
decreased in the vaccinated group. The PTB risk in cohort studies was significantly lower in
the vaccinated group (OR = 0.916, 95% CrI: 0.859?0.977, P = 99.5%) but adjusting for income
level of the country in cohort estimates made the association insignificant (Table 1, Fig 2(A),
Bayesian meta-analysis of LBW. Seventeen studies, including 2 RCTs and 15 cohort
studies (16 data sets were included from 15 cohort studies, as mentioned in the above section)
were included in the LBW analysis (? = 0.107, low heterogeneity). The overall results did not
satisfy the criterion of a P(OR<1) over 90% (OR = 0.928, 95% CrI: 0.432?2.112, P = 76.6%).
The results from RCTs likewise did not show a decreased LBW risk in the vaccinated group.
7 / 17
Fig 2. The results of meta-analyses of the associations between maternal influenza vaccination and adverse birth
outcomes: (A) preterm birth, (B) low birth weight, (C) small for gestational age, (D) congenital malformation, (E)
8 / 17
The risk of LBW in cohort studies was significantly lower in the vaccinated group (OR = 0.932,
95% CrI: 0.864?0.998, P = 97.8%). However, adjusting for covariates by meta-regression
nullified the significance in the analyses of both covariates (Table 1, Fig 2(B), S4(B) Table).
Bayesian meta-analysis of SGA. One RCT and 23 cohort studies (24 data sets were
included from 22 cohort studies as mentioned in the above section) were included in the
SGA analysis (? = 0.123, low heterogeneity). The overall results for the risk of SGA were not
significant (OR = 0.971, 95% CrI: 0.249?4.217, P = 63.3%). The summary results from RCTs
did not show significance. However, the cohort studies showed a significantly lower risk of
SGA (OR = 0.976, 95% CrI: 0.947?1.004, P = 95.5%) in the vaccinated group than in the
unvaccinated group. However, after adjusting for each covariate, the probabilities of a
decreased risk of SGA were considerably reduced and became insignificant (Table 1, Fig
2(C), S4(C) Table).
Bayesian meta-analysis of congenital malformations. Nineteen studies, comprising 1
RCT, 17 cohort studies, and 1 case-control study, were included in the congenital
malformation analysis (? = 0.100, low heterogeneity). The overall summary results of congenital
malformation presented a trend for higher risk in the vaccinated group than in the unvaccinated
group, but without significance (OR = 1.026, 95% CrI: 0.687?1.600, P = 38.0%). None of the
summary results from RCTs, cohort studies, and case-control studies were significant. No
covariate (season or income level) significantly affected the association (Table 1, Fig 2(D),
S4(D) Table). Since the information on the gestational period when the vaccination was given
was only available for congenital malformation, it was analyzed as covariate for this birth
outcome. Gestational period also did not have any influence on the risk of congenital
malformation. The overall summary effect after adjusting for gestation periods was as follows; OR:
0.865, 95% CrI: 0.192?2.757, P = 61.8% (S5 Table).
Bayesian meta-analysis of fetal death. Nineteen studies were included in the analysis of
fetal death, including 2 RCTs, 14 cohort studies, and 3 case-control studies (? = 0.165, low
heterogeneity). The overall risk of fetal death was not significantly decreased in the vaccinated
group (OR = 0.942, 95% CrI: 0.560?1.954, P = 61.6%). The summary results from RCTs and
case-control studies did not show a significant association. However, cohort studies showed
a significant reduction of the fetal death risk (OR = 0.822, 95% CrI: 0.718?0.931, P = 99.6%).
Meta-regression on covariates did not modify the final association (Table 1, Fig 2(E), S4(E)
In this study, we found that inactivated influenza vaccination during pregnancy was neither
associated with nor protective against the adverse birth outcomes of PTB, LBW, SGA, fetal
death, and congenital malformations. We summarized results from 48 studies analyzing over 2
million subjects (pregnant women and infants). Overall, no association was found, except for
the summary estimates of only cohort studies, which suggested protective effects against PTB
(after adjusting for season at the time of vaccination) and fetal death(after adjusting for each
covariate; season at the time of vaccination and countries? income level). The most notable
finding of this study is that we did not replicate previous findings of the beneficial effects of
maternal influenza vaccination on PTB, LBW, and fetal death, which a few previous systematic
reviews and meta-analyses have reported [
39, 47, 48
]. Additionally, we found that maternal
influenza vaccination was not associated with the incidence of any adverse birth outcomes,
which goes hand in hand with ensuring the safety of influenza vaccination during pregnancy.
The safety of maternal vaccination has always been a sensitive issue, and a report of a two-fold
increased risk of spontaneous abortion (SAB) in vaccinated women in a case-control study
9 / 17
conducted by Donahue et al. has recently received attention . That study triggered renewed
arguments about the safety of maternal influenza vaccination. Donahue et al. reported an
increased risk of spontaneous abortion (adjusted OR = 2.0; 95% CI, 1.1?3.6) in women who
received an influenza vaccine 1?28 days before the diagnosis of spontaneous abortion. In
response to the subsequent criticism and debates about their report, the authors defended
their findings, stating that despite not being able to escape from the limitation of residual
confounding, the considerable robustness of their study design implied that the increased SAB
risk found in their study was not a false positive[
]. Although we could not evaluate the risk
of fetal death in the period examined by Donahue et al. due to limitations in the available
information, we still analyzed the most recent high-quality publications, and on that basis might
suggest that the inactivated influenza vaccine administered during pregnancy did not increase
the risk of fetal death.
Two randomized placebo-controlled studies investigated the efficacy and safety of maternal
influenza vaccination in South Africa and South Asia, respectively. Madhi et al. (2014) [
reported higher rates of stillbirth (1.4% vs. 0.9%), PTB (10.5% vs. 9.4%), and LBW (13% vs.
12%) in the vaccinated group than in the unvaccinated group, but did not present statistical
significance. Steinhoff et al. (2017) [
] reported that maternal influenza vaccination reduced
the LBW rate by 15% (95% CI, 3%?25%) but did not significantly affect the rate of SGA. They
also reported low proportions of stillbirth (1.7% vs. 1.8%) and congenital defects (1.0% vs.
1.1%), but did not specify the significance.
The meta-analysis of Nunes et al. (2016), comprising 18 observational studies [
that inactivated influenza vaccination during pregnancy had protective effects on PTB and
LBW. Inactivated influenza vaccine reduced the risk of PTB and LBW by about 8%-13% and
12%-26%, respectively [
]. Another systematic review by Fell et al. (2014) included 1 RCT
and 26 observational studies. They reported a modestly reduced risk of fetal death and PTB,
noting the need for a cautious interpretation of the protective effect due to the shortcomings of
observational studies [
]. Bratton et al. (2014) assessed 7 observational articles on fetal death
(stillbirth or spontaneous abortion). They concluded that vaccinated women had about a 27%
lower risk of stillbirth than unvaccinated women .
However, McMillan et al. (2015) reported that influenza vaccination during pregnancy was
not associated with the risk of fetal death and spontaneous abortion, in agreement with our
findings. No associations were found for congenital malformations in women vaccinated
during their first trimester . Polyzos et al. (2015) assessed the risk of congenital anomalies
after influenza vaccination in 14 cohort studies and 1 case-control study. They also reported
no association for major malformations or congenital defects, regardless of the trimester,
agreeing with our study . The most recent cohort study, conducted by Getahun et al.
(2019) (n = 247,036) included the highest number of cases (n = 130,996), and reported that
vaccination was associated with a decreased risk of stillbirth (OR: 0.88, 95% CI: 0.78?0.99), but
no association was found for PTB and SGA[
]. Among the individual observational studies,
PTB was the most frequently examined outcome; Chamber et al. (2013) reported an increased
risk, whereas Olsen et al. (2016), Cleary et al. (2014) and Kallen et al. (2012) showed decreased
risks. None of the studies showed increased risks of LBW, but Dodds et al. (2012), Kallen et al.
(2012), and Legge et al. (2014) presented decreased risks of LBW. Fell et al. (2012) reported a
decreased risk of SGA, and Regan et al. (2016), Fell et al. (2012) and Pasternak et al. (2012)
showed significant (>50%) decreases in fetal death. However, none of the studies showed
significant findings for congenital malformation.
In the LBW and SGA meta-regression analyses in our study, the significant summary
estimates from cohort studies alone became null effects after adjusting for season at vaccination
and income level of the country. These results might be attributable to temperature exposure
10 / 17
(cold season) [
] since seasonal influenza vaccination is usually administered during late
fall to winter, along with nutritional and medical support for pregnancy outcomes .
A consistent null association between maternal influenza vaccination and adverse birth
outcomes was confirmed in this study, and the need for a cautious interpretation of the beneficial
birth outcomes reported by previous observational cohort studies was underscored [
recent WHO report also concluded that the evidence for maternal immunization reducing
adverse birth outcomes is conflicting. Numerous observational studies, as well as
meta-analyses and systematic reviews including those observational studies, cannot escape the possibility
of substantial bias, which is the main reason why the WHO report did not present a definitive
statement on maternal immunization .
Strengths and limitations
A major strength of this study is that it presents an up-to-date literature review, including the
highest number of reasonable-quality articles. In addition, this study did not restrict the study
design in the search step (only excluding cross-sectional studies), so we included RCTs, cohort
studies, and case-control studies. We applied a Bayesian 3-level random-effects model that
considered study design hierarchy to synthesize the adverse birth outcomes from different
types of study design. Through this method, we obtained different results for adverse birth
outcomes from the above-mentioned traditional meta-analyses [
39, 47, 48
], and compared the
summary estimates across different study designs. Additionally, when we conducted a
traditional meta-analysis comprising all cohort studies included in this analysis, the results were
different from the Bayesian meta-analysis for cohort studies only. The traditional
meta-analysis for cohort studies showed that the risks for PTB (OR = 0.91, 95% CI: 0.85?0.98) and fetal
death (OR = 0.81, 95% CI: 0.72?0.90) were significantly reduced in the vaccinated group, but
the risks for LBW(OR = 0.93, 95% CI: 0.87?1), SGA (OR = 0.97, 95% CI: 0.94?1.01) and
congenital malformations (OR = 1.02, 95% CI: 0.97?1.09) were not (S6 Table). Our up-to-date
Bayesian meta-analysis comprising only cohort studies presented significantly (P(OR<1) of
over 90%) reduced risk for PTB, LBW, SGA, and fetal death, but after adjusting for season and
income level as covariates, only fetal death remained significant. These results have the notable
implication that different statistical models yield different results, so we should be cautious in
interpreting the results.
Bayesian methods are becoming more common in a number of areas of healthcare research,
including meta-analyses . Using this type of model is highly advantageous for overcoming
heterogeneity across studies. The advantages of Bayesian methods are that past experience
or expert opinion can be used as prior information and the quantity of interest such as the
credible interval (CrI) or the hypothesis can be directly interpreted as the probability based on
the posterior distribution. In addition, Bayesian methods do not require a large sample size
assumption, so they can provide accurate inferential results when the number of samples is
]. Furthermore, Bayesian methods can improve the precision by using a hierarchical
model. However, in random-effects models, precision decreases with increasing heterogeneity,
and confidence intervals widen correspondingly[
]. If other studies with different 3-level
study designs, as used in our study, were to be included in a meta-analysis, the width of the
resulting confidence interval might be increased in comparison to that of each individual
Although we tried to overcome study design heterogeneity by using Bayesian methods, this
study did include many cohort studies, making it vulnerable to biases and making our study
somewhat dependent on the outcomes from cohort studies. However, as explicitly stated
throughout the Discussion section, when we compared our summary estimates to those of
11 / 17
cohort studies, different conclusions were drawn. Therefore, our new Bayesian meta-analysis
has novel implications. Another limitation is that incomplete information on vaccine type, the
exact perinatal period, and the season when the vaccine was administered hindered us
conducting a fully comprehensive covariate analysis; even though we gathered many variables in
the primary data acquisition steps, we could not incorporate these variables fully in the
metaregression. The presence of few studies from low/middle-income countries is another
limitation. Since few adverse birth outcomes occur in high-income countries, the small amount of
data from low/middle-income countries cannot provide convincing evidence whether
maternal influenza vaccination plans should be primarily implemented in low-resource countries.
Post-marketing RCT studies might be conducted, as they have been suggested by the Food and
Drug Administration as a maternal vaccination benefit-risk assessment tool when clinical trials
are not appropriate before marketing authorization [
In conclusion, this research presented additional evidence that maternal influenza
vaccination is not associated with harmful effects on birth outcomes. The beneficial effects reported
from previous studies on preterm birth, low birth weight, and fetal death could not be
S1 Table. Characteristics of studies included in the Bayesian meta-analysis.
S2 Table. Risk of bias assessment for included observational studies by the
S3 Table. Coefficients of meta-regression analyses on covariate effects.
S4 Table. Information on the studies included for each birth outcome: (A) PTB, (B) LBW,
(C) SGA, (D) congenital malformation, (E) fetal death.
S5 Table. Meta-regression results on congenital malformation adjusted by gestational
period when vaccination given.
S6 Table. Summary estimates of conventional meta-analysis with random effects model
constituted by cohort studies (A) PTB, (B) LBW, (C) SGA, (D) congenital malformation,
(E) fetal death.
S1 Fig. Risk of bias assessment for randomized controlled studies.
S2 Fig. Publication bias for included studies: (A) funnel plot of preterm birth, (B) funnel
plot of low birth weight, (C) funnel plot of small for gestational age, (D) funnel plot of
congenital malformation, (E) funnel plot of fetal death.
12 / 17
We would like to express our sincere appreciation to Myungin Shin (MS) and JungEun Lee
(JL) for their assistance in searching, selecting, and assessing the risk of bias of each included
study and to Hyemin Cho (HC) for assisting in the search for additional studies.
Conceptualization: Sohyun Jeong, Sunmee Jang.
Data curation: Sohyun Jeong, Junwoo Jo.
Formal analysis: Eun Jin Jang, Junwoo Jo.
Funding acquisition: Sunmee Jang.
Investigation: Sunmee Jang.
Methodology: Sohyun Jeong, Eun Jin Jang, Junwoo Jo, Sunmee Jang.
Project administration: Sohyun Jeong, Sunmee Jang.
Software: Eun Jin Jang, Junwoo Jo.
Supervision: Sunmee Jang.
Visualization: Eun Jin Jang, Junwoo Jo.
Writing ? original draft: Sohyun Jeong, Eun Jin Jang.
Writing ? review & editing: Sohyun Jeong, Eun Jin Jang.
13 / 17
14 / 17
15 / 17
Bratton K, Wardle M, Orenstein W, Omer S. Maternal influenza immunization and birth outcomes of
stillbirth and spontaneous abortion: a systematic review and meta-analysis. Clin Infect Dis 2015; 60:e11?9.
https://doi.org/10.1093/cid/ciu915 PMID: 25409473
Polyzos KA, Konstantelias AA, Pitsa CE, Falagas ME. Maternal Influenza Vaccination and Risk for
Congenital Malformations A Systematic Review and Meta-analysis. OBSTETRICS & GYNECOLOGY
Donzelli A. Influenza Vaccinations for All Pregnant Women? Better Evidence Is Needed. Int J Environ
Res Public Health. 2018; 15(9).
16 / 17
1. Siston A , Rasmussen S , Honein M , Fry A , Seib K , Callaghan W , et al. Pandemic 2009 influenza A (H1N1) virus illness among pregnant women in the United States . JAMA . 2010 ; 303 : 1517 - 25 . https:// doi.org/10.1001/jama. 2010 .479 PMID: 20407061
2. Izurieta H , Thompson W , Kramarz P , Shay D , Davis R , DeStefano F , et al. Influenza and the rates of hospitalization for respiratory disease among infants and young children . N Engl J Med . 2000 ; 342 : 232 - 9 . https://doi.org/10.1056/NEJM200001273420402 PMID: 10648764
3. Bhat N , Wright J , Broder K , Murray E , Greenberg M , Glover M , et al. Influenza-associated deaths among children in the United States, 2003 - 2004 . N Engl J Med . 2005 ; 353 : 2559 - 67 . https://doi.org/10. 1056/NEJMoa051721 PMID: 16354892
4. Zaman K , Roy E , Arifeen SE , Rahman M , Raqib R , Wilson E , et al. Effectiveness of Maternal Influenza Immunization in Mothers and Infants. The new england journal of medicine . 2008 ; 359 : 1555 - 64 . https:// doi.org/10.1056/NEJMoa0708630 PMID: 18799552
5. World Health Organization. Vaccines against influenza . WHO position paper-November 2012 2012. http://www.who.int/entity/wer/2012/wer8747.pdf?ua= 1 .
6. Rasmussen S , Watson A , Kennedy E , Broder K , Jamieson D . Vaccines and pregnancy . Past, present, and future. Semin Fetal Neonatal Med . 2014 ; 19 : 161 - 9 . https://doi.org/10.1016/j.siny. 2013 . 11 .014 PMID: 24355683
7. Sperling R , Engel S , Wallenstein S , Kraus T , Garrido J , Singh T , et al. Immunogenicity of trivalent inactivated influenza vaccination received during pregnancy or postpartum . Obstet Gynecol . 2012 ; 119 : 631 - 9 . https://doi.org/10.1097/AOG.0b013e318244ed20 PMID: 22353963
8. Jackson L , Patel S , Swamy G , Frey S , Creech C , Munoz F , et al. Immunogenicity of an inactivated monovalent 2009 H1N1 influenza vaccine in pregnant women . J Infect Dis . 2011 ; 204 : 845 - 63 .
9. Poehlling K , Szilagyi P , Staat M. Impact of maternal immunization on influenza hospitalizations in infants . Am J Obstet Gynecol . 2011 ; 204 ( 6 suppl 1 ): S141 - S8 . https://doi.org/10.1016/j.ajog. 2011 . 02 . 042 PMID: 21492825
10. Madhi SA , Cutland CL , Kuwanda L , Weinberg A , Hugo A , Jones S , et al. Influenza Vaccination of Pregnant Women and Protection of Their Infants. The New England Journal of Medicine . 2014 ; 371 : 918 - 31 . https://doi.org/10.1056/NEJMoa1401480 PMID: 25184864
11. Tapia MD , Sow SO , Tamboura B , Te?guete? I, Pasetti MF , Kodio M , et al. Maternal immunisation with trivalent inactivated influenza vaccine for prevention of infl uenza in infants in Mali:a prospective, activecontrolled, observer-blind, randomised phase 4 trial . Lancet Infect Dis . 2016 ; 16 ( 9 ): 1026 - 35 . https:// doi.org/10.1016/S1473- 3099 ( 16 ) 30054 - 8 PMID: 27261067
12. Steinhoff MC , Katz J , Englund JA , Khatry SK , Shrestha L , Kuypers J , et al. Year-round influenza immunisation during pregnancy in Nepal:a phase 4, randomised, placebo-controlled trial . Lancet Infect Dis . 2017 ; 17 ( 9 ): 981 - 9 . https://doi.org/10.1016/S1473- 3099 ( 17 ) 30252 - 9 PMID: 28522338
13. Nunes MC , Cutland CL , Jones S , Downs S , Weinberg A , Ortiz JR , et al. Efficacy of Maternal Influenza Vaccination Against All-Cause Lower Respiratory Tract Infection Hospitalizations in Young Infants: Results From a Randomized Controlled Trial . Clinical Infectious Diseases . 2017 . Epub may 29. https:// doi.org/10.1093/cid/cix497 PMID: 28575286
14. Lau J , Cai Y , Tsui H , Choi K. Prevalence of influenza vaccination and associated factors among pregnant women in Hong Kong . Vaccine . 2010 ; 28 : 5389 - 97 . https://doi.org/10.1016/j.vaccine. 2010 . 05 .071 PMID: 20542072
15. Goldfarb I , Panda B , Wylie B , Riley L . Uptake of influenza vaccine in pregnant women during the 2009 H1N1 influenza pandemic . Am J Obstet Gynecol . 2011 ; 204 ( 6 suppl 1 ): S112 - 5 . https://doi.org/10. 1016/j.ajog. 2011 . 01 .007 PMID: 21345408
16. Wiley K , Leask J . Respiratory vaccine uptake during pregnancy . The Lancet Respiratory Medicine . 2013 1: 1 - 3 .
17. Drees M , Johnson O , Wong E , Stewart A , Ferisin S , Silverman P . Acceptance of 2009 H1N1 influenza vaccine among pregnant women in Delaware . Am J Perinatol . 2012 ; 29 : 289 - 94 . https://doi.org/10. 1055/s-0031-1295660 PMID: 22147638
18. Pasternak B , Svanstrom H , M?lgaard-Nielsen D , Krause TG , Emborg H-D , M elbye M , et al. Risk of Adverse Fetal Outcomes Following Administration of a Pandemic Influenza A(H1N1) Vaccine During Pregnancy . JAMA . 2012 ; 308 ( 2 ): 165 - 74 . https://doi.org/10.1001/jama. 2012 .6131 PMID: 22782418
19. Conlin AMS , Bukowinski AT , Sevick CJ , DeScisciolo C , Crum-Cianflone NF . Safety of the Pandemic H1N1 Influenza Vaccine Among Pregnant U.S. Military Women and Their Newborns . OBSTETRICS & GYNECOLOGY . 2013 ; 121 ( 3 ): 511 - 8 .
20. Ludvigsson JF , Zugna D , Cnattingius S , Richiardi L , Ekbom A , Ortqvist A , et al. Influenza H1N1 vaccination and adverse pregnancy outcome . Eur J Epidemiol . 2013 ; 28 : 579 - 88 . https://doi.org/10.1007/ s10654-013-9813-z PMID: 23715672
21. Cleary BJ , Rice U ?, Eogan M , Metwally N , McAuliffe F. 2009 A/H1N1 influenza vaccination in pregnancy: uptake and pregnancy outcomes-a historical cohort study . European Journal of Obstetrics & Gynecology and Reproductive Biology . 2014 ; 178 : 163 - 8 .
22. Beau AB , Hurault-Delarue C , Vidal S , Guitard C , Vayssi?re C , Petiot D , et al. Pandemic A/H1N1 influenza vaccination during pregnancy: A comparative study using the EFEMERIS database . Vaccine . 2014 ; 32 : 1254 - 8 . https://doi.org/10.1016/j.vaccine. 2014 . 01 .021 PMID: 24486369
23. Nordin JD , Kharbanda EO , Benitez GV , Lipkind H , Vellozzi C , DeStefano F , et al. Maternal Influenza Vaccine and Risks for Preterm or Small for Gestational Age Birth . THE JOURNAL OF PEDIATRICS . 2014 ; 164 ( 5 ): 1051 - 7 . https://doi.org/10.1016/j.jpeds. 2014 . 01 .037 PMID: 24582484
24. Baum U , Leino T , Gissler M , Kilpi T , Jokinen J . Perinatal survival and health after maternal influenza A (H1N1)pdm09 vaccination: A cohort study of pregnancies stratified by trimester of vaccination . Vaccine . 2015 ; 33 : 4850 - 7 . https://doi.org/10.1016/j.vaccine. 2015 . 07 .061 PMID: 26238723
25. Kharbanda EO , Vazquez-Benitez G , Romitti PA , Naleway AL , Cheetham TC , Lipkind HS , et al. First Trimester Influenza Vaccination and Risks for Major Structural Birth Defects in Offspring . THE JOURNAL OF PEDIATRICS . 2017 ; 187 : 234 - 9 .e4. https://doi.org/10.1016/j.jpeds. 2017 . 04 .039 PMID: 28550954
26. McHugh L , Andrews RM , Lambert SB , Viney KA , Wood N , Perrett KP , et al. Birth outcomes for Australian mother-infant pairs who received an influenza vaccine during pregnancy, 2012 - 2014 : The FluMum study . Vaccine . 2017 ; 35 : 1403 - 9 . https://doi.org/10.1016/j.vaccine. 2017 . 01 .075 PMID: 28190746
27. Zerbo O , Qian Y , Yoshida C , Fireman BH , Klein NP , Croen LA . Association Between Influenza Infection and Vaccination During Pregnancy and Risk of Autism Spectrum Disorder . JAMA Pediatr . 2017 ; 171 ( 1 ):e163609. https://doi.org/10.1001/jamapediatrics. 2016 .3609 PMID: 27893896
28. Eick AA , Uyeki TM , Klimov A , Hall H , Reid R , Santosham M , et al. Maternal Influenza Vaccination and Effect on Influenza Virus Infection in Young Infants . Arch Pediatr Adolesc Med . 2011 ; 165 ( 2 ): 104 - 11 . https://doi.org/10.1001/archpediatrics. 2010 .192 PMID: 20921345
29. Heikkinen T , Young J , Beek Ev , Franke H , Verstraeten T , Weil JG , et al. Safety of MF59-adjuvanted A/ H1N1 influenza vaccine in pregnancy: a comparative cohort study . American Journal of Obstetrics&Gynecology . 2012 ; 207 : 177 . e1 - e8 .
30. Mackenzie IS , MacDonald TM , Shakir S , Dryburgh M , Mantay BJ , McDonnell P , et al. Influenza H1N1 (swine flu) vaccination: a safety surveillance feasibility study using self-reporting of serious adverse events and pregnancy outcomes . British Journal of Clinical Pharmacology . 2011 ; 73 ( 5 ): 801 - 11 .
31. Ludvigsson JF , Stro?m P , Lundholm C , Cnattingius S , Ekbom A , O? rtqvist ? , et al. Maternal vaccination against H1N1 influenza and offspring mortality: population based cohort study and sibling design . BMJ . 2015 ; 351 ( h5585 ): 1 - 6 .
32. Ludvigsson JF , Strom P , Lundholm C , Cnattingius S , Ekbom A , Ortqvist ? , et al. Risk for Congenital Malformation With H1N1Influenza Vaccine: A Cohort Study With Sibling Analysis . American College of Physicians. 2016 ; 165 : 848 - 55 .
33. Sugimura T , Nagai T , Kobayashi H , Ozaki Y , Yamakawa R , Hirata R . Effectiveness of maternal influenza immunization in young infants in Japan . Pediatrics International. 2016 ; 58 : 709 - 13 . https://doi.org/ 10.1111/ped.12888 PMID: 26670462
34. Chambers CD , Johnson DL , Xub R , Luo YJ , Louik C , Mitchell AA , et al. Safety of the 2010-11 , 2011 - 12 , 2012 - 13 , and 2013-14 seasonal influenza vaccines in pregnancy: Birth defects, spontaneous abortion, preterm delivery, and small for gestational age infants, a study from the cohort arm of VAMPSS . Vaccine. 2016 ; 34 : 4443 - 9 . https://doi.org/10.1016/j.vaccine. 2016 . 06 .054 PMID: 27449682
35. Louik C , Kerr S , Bennekom CMV , Chambers C , Jones KL , Schatz M , et al. Safety of the 2011-12 , 2012 - 13 , and 2013-14 seasonal influenza vaccines in pregnancy: Preterm delivery and specific malformations, a study from the case-control arm of VAMPSS . Vaccine. 2016 ; 34 : 4450 - 9 . https://doi.org/10. 1016/j.vaccine. 2016 . 06 .078 PMID: 27452865
36. Zerbo O , Modaressi S , Chan B , Goddard K , Lewis N , Bok K , et al. No association between influenza vaccination during pregnancy and adverse birth outcomes . Vaccine . 2017 ; 35 : 3186 - 90 . https://doi.org/ 10.1016/j.vaccine. 2017 . 04 .074 PMID: 28483192
37. Ahrens KA , Louik C , Kerr S , Mitchell AA , Werler MM . Seasonal Influenza Vaccination during Pregnancy and the Risks of Preterm Delivery and Small for Gestational Age Birth . Paediatr Perinat Epidemiol . 2014 ; 28 ( 6 ): 498 - 509 . https://doi.org/10.1111/ppe.12152 PMID: 25331380
38. Omer SB , Goodman D , Steinhoff MC , Rochat R , Klugman KP , Stoll BJ , et al. Maternal Influenza Immunization and Reduced Likelihood of Prematurity and Small for Gestational Age Births: A Retrospective Cohort Study . PLoS Medicine . 2011 ; 8 ( 5 ):e1000441. https://doi.org/10.1371/journal.pmed. 1000441 PMID: 21655318
39. Fell DB , Bhutta ZA , Hutcheon JA , Karron RA , Knight M , Kramer MS , et al. Fetal death and preterm birth associated with maternal influenza vaccination: systematic review . BJOG . 2015 ; 122 : 17 - 26 . https://doi. org/10.1111/ 1471 - 0528 .12977 PMID: 25040307
40. Dodds L , MacDonald N , Scott J , Spencer A , Allen VM , McNeil S . The Association Between Influenza Vaccine in Pregnancy and Adverse Neonatal Outcomes . J Obstet Gynaecol Can . 2012 ; 34 ( 8 ): 714 - 20 . https://doi.org/10.1016/S1701- 2163 ( 16 ) 35336 - 1 PMID: 22947404
41. Sammon CJ , Snowball J , McGrogan A , Vries CSd . Evaluating the Hazard of Foetal Death following H1N1 Influenza Vaccination; A Population Based Cohort Study in the UK GPRD . PLoS ONE . 2012 ; 7 ( 12 ):e51734. https://doi.org/10.1371/journal.pone. 0051734 PMID: 23341865
42. Kallen B , Olausson P . Vaccination against H1N1 influenza with Pandemrix during pregnancy and delivery outcome: a Swedish register study . BJOG . 2012 ; 119 : 1583 - 90 . https://doi.org/10.1111/j.1471- 0528 . 2012 . 03470 . x PMID : 22901103
43. Pasternak B , Svanstro?m H , M?lgaard-Nielsen D , Krause TG , Emborg H-D , M elbye M , et al. Vaccination against pandemic A/H1N1 2009 influenza in pregnancy and risk of fetal death: cohort study in Denmark . BMJ . 2012 ; 344 :e2794. https://doi.org/10.1136/bmj.e2794 PMID: 22551713
44. Sheffield JS , Greer LG , Rogers VL , Roberts SW , Lytle H , McIntire DD , et al. Effect of Influenza Vaccination in the First Trimester of Pregnancy . Obstet Gynecol . 2012 ; 120 : 532 - 7 . https://doi.org/10.1097/ AOG.0b013e318263a278 PMID: 22914461
45. Richards JL , Hansen C , Bredfeldt C , Bednarczyk RA , Steinhoff MC , Adjaye-Gbewonyo D , et al. Neonatal Outcomes After Antenatal Influenza Immunization During the 2009 H1N1 Influenza Pandemic: Impact on Preterm Birth, Birth Weight, and Small for Gestational Age Birth . Clinical Infectious Diseases . 2013 ; 56 ( 9 ): 1216 - 22 . https://doi.org/10.1093/cid/cit045 PMID: 23378281
46. Legge A , Dodds L , MacDonald NE , Scott J , McNeil S . Rates and determinants of seasonal influenza vaccination in pregnancy and association with neonatal outcomes . CMAJ . 2014 ; 186 ( 4 ): E157 - E64 . https://doi.org/10.1503/cmaj.130499 PMID: 24396098
47. Nunes MC , Aqil AR , Omer SB , Madhi SA . The Effects of Influenza Vaccination during Pregnancy on Birth Outcomes: A Systematic Review and Meta-Analysis . Am J Perinatol . 2016 ; 33 : 1104 - 14 . https:// doi.org/10.1055/s-0036-1586101 PMID: 27603545
McMillana M , Porritt K , Kralikc D , Costi L , Marshall H . Influenza vaccination during pregnancy: A systematic review of fetal death, spontaneous abortion, and congenital malformation safety outcomes . Vaccine . 2015 ; 33 : 2108 - 17 . https://doi.org/10.1016/j.vaccine. 2015 . 02 .068 PMID: 25758932
Vazquez-Benitez G , Kharbanda EO , Naleway AL , Lipkind H , Sukumaran L , McCarthy NL , et al. Risk of Preterm or Small-for- Gestational-Age Birth After Influenza Vaccination During Pregnancy: Caveats When Conducting Retrospective Observational Studies. Am J Epidemiol . 2016 ; 184 ( 3 ): 176 - 86 . https:// doi.org/10.1093/aje/kww043 PMID: 27449414
Donahue J , Kieke B , King J , DeStefano F , Mascola M , Irving S , et al. Association of spontaneous abortion with receipt of inactivated influenza vaccine containing H1N1pdm09 in 2010-11 and 2011-12 . Vaccine. 2017 ; 35 ( 40 ): 5314 - 22 . https://doi.org/10.1016/j.vaccine. 2017 . 06 .069 PMID: 28917295
Centers for Disease Control and Prevention (CDC). Flu Vaccination & Possible Safety Signal:Information & Guidance for Health Care Providers Atlanta , GA : CDC; 2017 . https://www.cdc.gov/flu/ professionals/vaccination/vaccination-possible -safety-signal .html.
Fell DB , Azziz-Baumgartner E , Baker MG , Batra M , Beaut e? J, Beutels P , et al. Influenza epidemiology and immunization during pregnancy: Final report of a World Health Organization working group . Vaccine . 2017 ; 35 : 5738 - 50 . https://doi.org/10.1016/j.vaccine. 2017 . 08 .037 PMID: 28867508
Moher D , Liberati A , Tetzlaff J , Altman D , PRISMA Group. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement . Ann Intern Med . 2009 ; 151 : 264 - 9 . https://doi. org/10.7326/ 0003 -4819-151-4- 200908180 -00135 PMID: 19622511
Sutton A , Cooper N , Lambert P , Jones D , Abrams K , Sweeting M . Meta-analysis of rare and adverse event data . Expert Rev Pharmacoecon Outcomes Res . 2002 ; 2 ( 4 ): 367 - 79 . https://doi.org/10.1586/ 1473722.214.171.1247 PMID: 19807443
Sutton AJ , Abrams KR . Bayesian methods in meta-analysis and evidence synthesis . Statistical Methods in Medical Research . 2001 ; 10 : 277 - 303 . https://doi.org/10.1177/096228020101000404 PMID: 11491414
Prevost T , Abrams K , Jones D. Using hierarchical models in generalised synthesis of evidence: an example based on studies of breast cancer screening . Statistics in Medicine. 2000 : 3359 - 76 . PMID: 11122501
Cochrane Library . 16 . 8 Bayesian and hierarchical approaches to meta-analysis . 2011. In: Cochrane Handbook for Systematic Reviews of Interventions [Internet]. London,UK: Cochran Library. 5.1.0.
Davies H , Crombie I , Tavakoli M. When can odds ratios mislead? BMJ. 1988 ; 316 ( 7136 ): 989 - 91 .
Stare J , Maucort-Boulch D. Odds Ratio , Hazard Ratio and Relative Risk . Metodoloski zvezki. 2016 ; 13 ( 1 ): 59 - 67 .
Symons M , Moore D. Hazard rate ratio and prospective epidemiological studies . J Clin Epidemiol . 2002 ; 55 ( 9 ): 893 - 9 . PMID: 12393077
Peter J , John P , Graham P , Morgan J , George I , Berstein A . Corticosteroids in the prevention and treatment of acute respiratory distress syndrome (ARDS) in adults:meta-analysis . BMJ . 2008 ; 336 : 1006 - 9 . https://doi.org/10.1136/bmj.39537.939039.BE PMID: 18434379
Brooks S , Gelman A. General Methods for Monitoring Convergence of Iterative Simulations . Journal of Computational and Graphical Statistics . 1998 ; 7 ( 4 ): 434 - 55 .
Sterne J , Gavaghan D , Egger M. Publication and related bias in metaanalysis: power of statistical tests and prevalence in the literature . Clin Epidemiol . 2000 ; 53 : 1119 - 29 .
Spiegelhalter D , Abrams K , Myles J . Bayesian approaches to clinical trials and health-care evaluation . Chichester, England: John Wiley & Sons, Ltd; 2004 .
World Bank Group. The World Bank: Data Washington, DC 20433 USA: The World Bank,; 2018 . https://data.worldbank.org/income-level/high-income.
Cochrane Library . Assessing Risk of Bias in Included Studies: Cochrane Library; 2008 . http://methods. cochranehttp://methods.org/bias/assessing -risk-bias-included-studies.
Wells G , Shea B , O'Connell D , Peterson J , Welch V , Losos M , et al. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses Ottawa , Canada: The Ottawa Hospital; 2018 .
71. Newcastle-Ottawa Quality Assessment Form for Cohort Studies . https://www.ncbi.nlm.nih.gov/books/ NBK115843/bin/appe-fm3. pdf .
72. Donahue JG , Kieke BA , King JP , DeStefano F , Mascola MA , Irving SA , et al. Association of spontaneous abortion with receipt of inactivated influenza vaccine containing H1N1pdm09 in 2010-11 and 2011-12 . Vaccine. 2017 ; 35 : 5314 - 22 . https://doi.org/10.1016/j.vaccine. 2017 . 06 .069 PMID: 28917295
73. Donahue JG , Kieke BA , King JP , Mascola MA , Belongia EA . Response to three Letters to the Editor regarding: Donahue JG , et al. ' 'Association of spontaneous abortion with receipt of inactivated influenza vaccine containing H1N1pdm09 in 2010-11 and 2011- 12 ?. Vaccine 35 ( 2017 ) 53 - 14 - 5322 . Vaccine. 2018 : 2231 - 2 . https://doi.org/10.1016/j.vaccine. 2017 . 12 .044 PMID: 29609919
74. Getahun D , Fassett MJ , Peltier MR , Takhar HS , Shaw SF , Im TM , et al. Association between seasonal influenza vaccination with pre- and postnatal outcomes . Vaccine . 2019 .
75. Rashid H , Kagami M , Ferdous F , Ma E , Terao T , Hayashi T , et al. Temperature during pregnancy influences the fetal growth and birth size . Tropical Medicine and Health . 2017 ; 45 ( 1 ). https://doi.org/10. 1186/s41182-016 -0041-6 PMID: 28077924
76. Ha S , Zhu Y , Liu D , Sherman S , Mendola P . Ambient temperature and air quality in relation to small for gestational age and term low birthweight . Environ Res . 2018 ; 155 : 394 - 400 .
77. WHO. WHA Global Nutrition Targets 2025 : Low Birth Weight Policy Brief . 2014 .
78. Savitz D , Fell D , Ortiz J , Bhat N. Does influenza vaccination improve pregnancy outcome? Methodological issues and research needs . Vaccine . 2015 ; 33 : 6430 - 5 . https://doi.org/10.1016/j.vaccine. 2015 . 08 . 041 PMID: 26319740
79. Sutton AJ , Abrams KR . Bayesian methods in meta-analysis and evidence synthesis . Statistical Methods in Medical Research 2001 ; 10 : 277 - 303 . https://doi.org/10.1177/096228020101000404 PMID: 11491414
80. Cochrane Library . 12 . 4 .1 Confidence Intervals . 2011 . In: Cochrane Handbook for Systematic Reviews of Interventions [Internet] . London,UK: Cochrane Library. 5 . 1 .0.
81. FDA. FDA Briefing Document: Vaccines and Related Biological Products Advisory Committee Meeting Silver Spring , MD 20993, USA: FDA; 2015 .