 Research Article
 Open Access
Fast and accurate dynamic estimation of field effectiveness of meningococcal vaccines
 Lorenzo Argante^{1, 2, 3}Email author,
 Michele Tizzoni^{2} and
 Duccio Medini^{3}
 Received: 19 February 2016
 Accepted: 10 June 2016
 Published: 30 June 2016
Abstract
Background
Estimating the effectiveness of meningococcal vaccines with high accuracy and precision can be challenging due to the low incidence of the invasive disease, which ranges between 0.5 and 1 cases per 100,000 in Europe and North America. Vaccine effectiveness (VE) is usually estimated with a screening method that combines in one formula the proportion of meningococcal disease cases that have been vaccinated and the proportion of vaccinated in the overall population. Due to the small number of cases, initial point estimates are affected by large uncertainties and several years may be required to estimate VE with a small confidence interval.
Methods
We used a Monte Carlo maximum likelihood (MCML) approach to estimate the effectiveness of meningococcal vaccines, based on stochastic simulations of a dynamic model for meningococcal transmission and vaccination. We calibrated the model to describe two immunization campaigns: the campaign against MenC in England and the Bexsero campaign that started in the UK in September 2015. First, the MCML method provided estimates for both the direct and indirect effects of the MenC vaccine that were validated against results published in the literature. Then, we assessed the performance of the MCML method in terms of time gain with respect to the screening method under different assumptions of VE for Bexsero.
Results
MCML estimates of VE for the MenC immunization campaign are in good agreement with results based on the screening method and carriage studies, yet characterized by smaller confidence intervals and obtained using only incidence data collected within 2 years of scheduled vaccination. Also, we show that the MCML method could provide a fast and accurate estimate of the effectiveness of Bexsero, with a time gain, with respect to the screening method, that could range from 2 to 15 years, depending on the value of VE measured from field data.
Conclusions
Results indicate that inference methods based on dynamic computational models can be successfully used to quantify in near real time the effectiveness of immunization campaigns against Neisseria meningitidis. Such an approach could represent an important tool to complement and support traditional observational studies, in the initial phase of a campaign.
Keywords
 Neisseria meningitidis
 Vaccine effectiveness
 Invasive meningococcal disease
 Meningococcal carriage
 Bexsero
 Monte Carlo maximum likelihood
 Computational dynamic models
Background
Neisseria meningitidis is an aerobic Gramnegative diplococcus that causes annually 1.2 million cases of meningitis and 135,000 deaths globally [1]. This humanrestricted opportunistic pathogen is part of the commensal flora that colonizes the upper respiratory tract of healthy individuals. N. meningitidis strains are divided into 12 serogroups, on the basis of the immunochemistry of their capsular polysaccharides [2]. Serogroups A, B, C, W, Y, and X account for most of the invasive disease cases worldwide; serogroup B (MenB) is the leading cause of meningococcal meningitis in Europe (90 %), New Zealand (82 %), Australia (80 %), Argentina (67 %), Japan (57 %), and Canada (53 %) [3–11].
Invasive meningococcal disease (IMD) has the highest fatality rate among other vaccinepreventable diseases after rabies [12] (up to 40 % for meningococcal septicemia [13]), is easily misdiagnosed [14] and can cause death in 24 hours. Up to 20 % of survivors exhibit permanent lifelong disabilities, including brain damage, deafness, kidney failure, and limb amputation [15]. All age groups are susceptible to IMD. Infants, however, are 17 times more likely to develop disease compared to the general population [16]. A smaller peak of IMD incidence is also observed in adolescents and young adults. Notably, more than 10 % of the general population asymptomatically carries N. meningitidis [17], and in the first 30 years of life, each person is expected to become a carrier of the meningococcus 10 times [18]. The age distribution of asymptomatic carriage, however, is markedly different from that of IMD. Carriage in infants is low, and grows slowly up to approximately 10 % in preadolescents. A sharp increase of carriage prevalence is observed after 15 years of age, reaching 25 % or more at the age of 20, and then it decreases slowly to approximately 10 % in the elderly [19].
Due to the ambiguous initial clinical manifestations and to the extremely rapid development of disease, vaccination is the only broadly effective measure against IMD [15]. Vaccines based on serogroup A, C, W, and Y capsular polysaccharide conjugates have been licensed in many parts of the world [20–23]. Among others, a national immunization campaign that started in 1999 in the UK showed the high efficacy of meningococcal C (MenC) conjugate vaccines against serogroupspecific disease and carriage [21, 24–30]. Only very recently two broadly protective MenB vaccines were licensed, Bexsero (GSK) and Trumenba (Pfizer), both licensed in the USA for persons between 10 and 25 years of age. Bexsero was also licensed in Europe, Canada, Australia, and elsewhere for individuals from 2 months of age and older and is being used in a national immunization program for infants and children in the UK that started in September 2015 [31]. Both vaccines were licensed based on safety and immunogenicity data. Although strain coverage was also assessed for Bexsero worldwide through the Meningococcal Antigen Typing System [32], indicating strain coverage between 66 and 91 % worldwide [33], no formal proof of field effectiveness is available yet.
Mass immunization campaigns can be evaluated in the field by observational studies, monitoring the number of cases emerging in vaccinated and nonvaccinated cohorts [34]. A number, called vaccine effectiveness (VE), measures how much a vaccine is efficacious in reducing the incidence of a certain disease in groups of vaccinated, when compared to the incidence in subjects that have not received the vaccine [34].
Case–control studies would represent the first choice when estimating meningococcal VE, since cohort studies are not feasible due to the low disease incidence [34]. However, IMD incidence is so low that also the applicability of case–control studies is severely limited. A popular alternative to case–control studies is the screening method [35]. This method is a variant of the case–control methods where, instead of choosing one or more individual controls per case, the entire population at risk is used as a reference group [36]. The screening method estimates the VE using a mathematical formula (see Additional file 1 for details) that combines three basic quantities, measured at one point in time: the reported number of disease cases, the number of vaccinated among cases, and the proportion of the overall population under study that has been vaccinated [35]. Compared to other observational methods, screening is considerably less resourceintensive, because all the information needed is usually available from surveillance systems and does not require specific designs. Additionally, it has the great advantage of being rapid, and this is why it has been proposed as a first step to determine VE [36]. On the other hand, the screening method is known to lack precision, its estimates being very sensitive to errors in the input data [37, 38].
For meningococcal vaccines, the screening method is the main candidate to estimate VE. It has been employed to evaluate the 1999 MenC immunization campaign in England [25], and it will likely be used to estimate Bexsero effectiveness from field data. However, in this context its advantages are significantly reduced by the very low incidence of the disease, which will inevitably lengthen the time needed to obtain a precise VE estimate. Simple statistics arguments [35] allow us to predict the number of IMD cases that will be needed to estimate some hypothetical true value of VE with a desired precision, that is with a 95 % confidence interval (CI), possibly bounded not too far from the point estimate. Using available incidence data [39], cases can be converted into observation time. For instance, assuming that the expected IMD incidence in the UK will be the same as that experienced during 2012–2014, on average, and assuming a VE of 60 % or higher, at least 15 years will be needed to estimate effectiveness with a lower bound of the 95 % CI higher than 45 % (see Additional file 1 for details).
An important role in meningococcal vaccines is played by the ability to confer or not a mucosal immunity that can protect people from N. meningitidis nasopharynx colonization, therefore reducing asymptomatic transmissions of invasive strains. Here, we will refer to it as the indirect effect of the vaccine because it can provide protection also to unvaccinated individuals, conferring herd immunity. This mechanism can strongly impact the transmission dynamics and reduce the morbidity of pathogens like N. meningitidis, as has been observed and quantified for vaccines against MenC [28, 30, 40]. However, by employing notified IMD cases only, the screening method fails to evaluate the indirect VE, since it contributes to lower the IMD incidence both in vaccinated as well as in unvaccinated individuals (details in Additional file 1).
It is possible to evaluate VE by comparing the disease incidence among unvaccinated individuals only with the disease incidence in the same population, measured before the vaccination campaign [34]. This has been done for the MenC conjugate vaccines in England [29], providing evidence of a herd immunity effect, but with low precision and large fluctuations around the point estimate across age groups. A more robust way to evaluate the indirect effectiveness is by measuring the meningococcal carriage, instead of the invasive disease, at two points in time. For this purpose, large crosssectional carriage studies have been conducted to quantify the meningococcal carriage in adolescents recruited in England before and after the vaccination campaign [28, 30]. However, such an approach is inevitably expensive and timeconsuming.
The challenge of shortening the time needed to evaluate the direct VE and, at the same time, providing a tool to improve the evaluation of the indirect effectiveness using notified cases only, motivated our work. For this aim, we devised a VE estimation method based on a Monte Carlo maximum likelihood (MCML) inferential approach that combines stochastic simulations of a computational dynamic model for N. meningitidis asymptomatic transmission and real case notification data. More specifically, we built a discrete mechanistic model that stochastically reproduces the dynamics of N. meningitidis infection and vaccination at the population level. The model structure is based on previous works [18, 41–43] and explicitly takes into account both the direct and indirect effect of the vaccine as two free parameters. Unlike previous works, where modeling was used to forecast the impact of different vaccination strategies in England, generating scenarios of future disease incidence based on hypothetical values of VE, here we adopted a reverse approach: we inferred the most probable values of VE via likelihood maximization given the time series of disease cases notified before and after the campaign start, using a sequential Monte Carlo (SMC) method, also known as particle filtering [44].
The main goal of our study was to estimate the direct effectiveness of meningococcal vaccines with the same accuracy as the screening method but faster, aiming at a better precision in a shorter time. Furthermore, we wanted to show that our modelbased approach allows for a simultaneous estimation of the indirect effectiveness, achieving similar results as carriage studies without using additional data beside IMD notifications and vaccine uptake.
Methods
In our work, we focused on two different mass immunization campaigns against meningococcal disease in England: the MenC campaign that started in 1999 and the MenB campaign that started in September 2015. In the following, we describe the model’s definition, the data used to parameterize it, and the MCML inference method used to estimate the VE. Further details on the methods are reported in Additional file 1.
Dynamic model of meningococcal transmission, carriage, and invasive disease
We defined a dynamical compartmental model that reproduces meningococcal carriage, transmission, invasive disease, and vaccination. In particular, we were interested in those characteristics that are relevant to the inference of (i) the VE in inducing protection from invasive disease, or direct effectiveness, and (ii) the VE in reducing the asymptomatic transmission, or indirect effectiveness. In the following, we will refer to them as VE_{dir} and VE_{ind}, respectively.

Birth. Newborns are added to the unvaccinated susceptibles of age 0, according to the natural birth rate.

Ageing. Individuals in a given age group move to the next age group, if not already in the last age class.

Death. Some individuals of a given age in every compartment die, according to the natural death rate.

Infection. Susceptibles become carriers depending on the force of infection λ_{ a }(t). If vaccinated immune, the probability of being infected is reduced by a factor corresponding to 1−VE_{ind}.

Recovery. Carriers spontaneously lose the carrier status with frequency ρ, whose inverse 1/ρ is the average duration of carriage τ_{car}.

Progression to IMD. In rare cases, susceptible individuals develop IMD 2–10 days after meningococcal carriage acquisition [17, 45, 46]. We model progression to IMD as an event alternative to the asymptomatic infection. It can happen only immediately after transmission, with an agedependent risk of IMD given infection, denoted as θ_{ a }(t).

Vaccination. Unvaccinated individuals in cohorts targeted by vaccination are moved to the vaccinated compartments according to the vaccine schedule and the population coverage γ_{ a }. Only a fraction VE _{dir} of the vaccinated gains immunity to IMD.

Waning of immunity. A fraction ω_{ a } of the vaccinated and immune spontaneously move to the vaccinated nonimmune compartment, after a time period τ_{imm} that depends on age and vaccine type (MenC or MenB vaccine).
The model is stochastic. The compartments and the transitions between them can be represented as a set of stochastic differential equations (see Additional file 1).
Once the model is parameterized to reproduce the epidemiology of meningococcal disease in the absence of vaccination campaigns, the effectiveness is inferred via MCML by means of two parameters, namely the VE against invasive disease VE_{dir} and against carriage acquisition VE_{ind}. Both are bounded between zero and one, but they act in very different ways. At each time step after the start of the vaccination campaign, VE_{dir} represents the fraction of vaccinated individuals who successfully gain vaccineinduced protection against IMD. Those individuals are, therefore, immune and cannot develop the IMD. The remaining fraction (1−VE_{dir}) of individuals go into the notimmune compartments and can develop the disease as if they were not vaccinated at all. In this sense, our modeling approach uses an allornothing vaccine [34].
On the other hand, VE_{ind} is a parameter that multiplies only the force of infection experienced by vaccinated and immune compartments, thus reducing the probability of meningococcal colonization, i.e., the transition SVI →CVI. It represents the ability gained by vaccinated subjects for developing a certain level of protection against N. meningitidis acquisition.
The parameters VE_{ind} and VE_{dir} are in fact inputs of the transmission model, which is otherwise completely specified by the other parameters and the initial compartment population. The outputs of the transmission model are the numbers of infection events per time step that can generate IMD cases (transitions S →C and SV →CV). We call these numbers J_{ a }(t) and JV_{ a }(t), respectively, for nonvaccinated and vaccinated of age a. They will be used to infer which values of VE will most likely lead to the reported time series of IMD cases. It is important to note that the number of infections experienced by immune individuals is not collected because in the model formulation, those infections cannot lead to IMD cases.
where the righthand side represents the probability that \(D^{\text {obs}}_{a,v}(y)\) IMD cases emerge from J_{a,v}(y) infection events, when the probability of an IMD given infection is θ_{ a }.
Model parameterization
Parameter values of the dynamic model
Description  Value  Source  

α  Yearly crude birthrates  Between 11.3 and 13.0 per 1000 population, variable by year  [47] 
μ _{ a }  Yearly mortality rates  Variable by age (Additional file 1: Figure S2)  [47] 
τ _{car}  Duration of carriage (1/ρ)  6 [3–9] months  
β _{ a }  Quarterly rates of infection given contacts with carriers  Variable by age (Additional file 1: Figure S4)  Fitted 
\(m_{aa'}\phantom {\dot {i}\!}\)  Daily contact rates matrix  Variable by age of subject and contact (Additional file 1: Figure S3)  [48] 
θ _{ a }  Quarterly risks of disease given infection  Variable by age (Additional file 1: Figure S5)  Fitted 
γ _{ a }  Quarterly fraction of population vaccinated  MenB: 95 % of 1 year olds MenC: variable by age and schedule  MenB: assumed MenC: [52] 
τ _{imm,a}  Duration of immunity to disease  MenB: 18 months MenC: variable by age (15 [9–24] mths, 5 years, 10 years)  
ω _{ a }  Waning of immunity fraction  MenB: 100 % MenC: 58.4 %  MenB: assumed MenC: [26] 
The model reproduces the English demographic structure and its temporal evolution, according to population estimates and annual birth and death rates, as reported by the UK Office of National Statistics [47].
The symbol \(\phantom {\dot {i}\!}m_{aa'}\) denotes the reported number of daily contacts between persons of age a and a^{′}, and it is assumed to follow contact patterns observed in Great Britain [48]. The carriage acquisition rate β_{ a } is a free parameter to be calibrated against serogroupspecific carriage data (see Additional file 1 for details).
Infected individuals lose their carrier status with probability ρ per time step and can be infected again. We set ρ=1/τ_{car} with τ_{car} representing the average duration of carriage. In our base case, we assumed τ_{car} to be equal to 6 months and independent from the serogroup under study. For a sensitivity analysis, we tested values between 3 and 9 months as reported in the literature [18, 41, 43, 45, 49–51] (see Additional file 1 for details).
During the immunization campaign, a fraction γ_{ a } of nonvaccinated individuals belonging to cohorts scheduled for vaccination moves to the vaccinated compartments. Only individuals who have received all the recommended doses are taken into account and they are moved according to the approximate dates of injections, for both routine and catchup implementation. The MenC campaign coverage data have been published [25, 41, 52]. For Bexsero, we use expected coverage data, following previous works [43, 53].
The vaccine confers protection for a limited time and the duration of vaccineinduced immunity τ_{imm} depends on the specific vaccine and the age of the vaccinated cohort. For the MenC campaign, the protection is set to last 15 months for the routine vaccination, 5 years for children vaccinated at 12 months, and 10 years for all those vaccinated at older ages [41]. For the MenB campaign, we assume infants to be routinely vaccinated at 2, 4, and 12 months, as announced by Public Health England (PHE) [54]. Estimates of the duration of protection for Bexsero are not available yet [55]. We assume the vaccineinduced protection to last at least 18 months after the booster dose, in line with previous works [43].
where the numbers \(D^{\text {exp}}_{a}\) are occurrences of IMD cases and the \(J^{\text {exp}}_{a}\) are the numbers of infection events produced by the model, both expected to emerge during a time step when no vaccination campaign is implemented. The D^{exp} are the yearly average numbers of laboratoryconfirmed IMD cases by age, reported during the last 1–3 epidemiological years before the vaccination campaign. The numbers \(J^{\text {exp}}_{a}\) are calculated from the infection term of the model’s equations set at the initial time step (see Additional file 1). The risks θ_{ a } of IMD given infection by age are fixed, implying the assumption that the pathogenicity of meningococci would remain constant in the absence of immunization campaigns.
Inference of VE
The MCML procedure that we use to infer the parameters VE_{dir} and VE_{ind} is based on a SMC approach, also known as particle filtering [44]. This method was first introduced in epidemiology to estimate the environmental contribution to cholera transmission [58], and then applied to a variety of epidemic models [59–61]. Here, the likelihood of the combination of VEs is calculated given empirical observations, which are time series of the number of cases by age group notified during each epidemiological year after vaccination start. The observables are connected to the abovedescribed transmission model by means of the risks θ_{ a }.
The particle filtering procedure allows us to sample the trajectories probabilistically in the space of the J, taking into account the history of cases and the model structure.
The best estimates of VE_{dir} and VE_{ind} correspond to the maximum of the likelihood. Likelihood profiling allows to calculate 95 % CIs around it.
MCML method settings for the MenC vaccination in England
In the first part of our work, we simulated the mass immunization campaign against MenC disease that started in England at the end of 1999. To calibrate the model, England’s population is initially set to mid1998 estimates. The agedependent distribution of MenC IMD cases \(D^{\text {exp}}_{a}\) used to calibrate the method is the one reported during 1998 and the first half of 1999, then normalized to reproduce, on average, the total number of cases notified during the epidemiological year 1998/99. As a sensitivity analysis, we tested different \(D^{\text {exp}}_{a}\) by varying the average total number of cases generated by the model, while keeping a constant age distribution (see Additional file 1: Section 5). We normalized the carriage prevalence distribution by age to reproduce the prevalence of serogroup C samples observed in carriage studies before 1999 [25]. The SMC algorithm was run using 1000 particles and varying the values of VE_{dir} and VE_{ind} between 0 % and 100 % with steps of 0.1 % to build the yearly likelihood surfaces.
Simulation of MenB vaccine campaign in England and MCML settings
In the second part of our work, we simulated the mass immunization campaign against MenB disease that started in England in September 2015. The initial population of England employed for calibration is the one estimated in 2014. To calculate the expected occurrences of MenB cases \(D^{\text {exp}}_{a}\), we used the agedependent distribution of MenB IMD cases reported during the epidemiological year 2007/08, then normalized to produce, on average, the total number of cases notified every year from mid2012 to mid2015. The carriage prevalence distribution by age was normalized to reproduce the prevalence of serogroup B samples observed in carriage studies [30].
In this scenario, disease case occurrences after the campaign start \(D^{\text {obs}}_{a,v}\) are not available yet. To test our approach, we used the model in a generative way, thus simulating future incidence scenarios with constant values of VE_{dir} and VE_{ind}, as usually done to predict the impact or the costeffectiveness of vaccination campaigns [41, 43, 62]. In detail, we assumed VE_{ind} to be always equal to 0 %, to be as conservative as possible, since reliable estimates of the indirect effect of Bexsero are not available. Then, we assumed four possible values of VE_{dir}: 60 %, 70 %, 80 %, and 90 %. For each of the four scenarios, we ran the model 10,000 times, thus producing 10,000 trajectories in the space of the MenB disease case occurrences, stratified by age and vaccination status. Then, we selected the median trajectory to be the reference time series of synthetic MenB cases observed after vaccination.
Given the simulated IMD incidence curve, we used the MCML to estimate the most likely value of VE_{dir} and the corresponding CIs at different years, by assuming a constant VE_{ind}. In each scenario, the SMC algorithm was run using 1000 particles and varying the values of VE_{dir} between 0 % and 100 % with steps of 0.1 % to build the yearly likelihood curves.
Finally, we compared the results obtained with the MCML method against the screening method’s sample size analysis [35]. In particular, for eight consecutive years after the start of the vaccination, we compared the MCML point estimate of VE_{dir} and its CI against the CI provided by the screening method, for the same value of VE_{dir}. We estimated the screening method’s CIs by converting the required sample of IMD cases given by the Farrington formula [35] to the time needed to observe such cases (see Additional file 1 for details). Here, we calculated that the baseline incidence of IMD would be 70 cases a year in the fully immunized cohort (12–23 months old), if no vaccination is implemented.
Results
 1.
The underlying calibration of the dynamic model allows us to reproduce realistically the epidemiology of meningococcal disease in England.
 2.
The MCML method allows us to estimate both direct and indirect VE using case notification data.
 3.
It provides realistic estimates of the VE compared to observational methods.
 4.
It is faster than the screening method in reaching the same precision around the point estimate of VE.
Here, we discuss these in detail.
The dynamic model is calibrated to reproduce meningococcal epidemiology realistically
We calibrated the model for simulating two different vaccination campaigns in England, against the serogroup C and B meningococcal disease, as described in “Methods”. In both cases, we tested the calibration by running the model for 50 years under the basic transmission dynamics without any vaccination campaign.
The model reproduces a realistic demographic evolution of the population in England, with fluctuations due to population differences across age groups (see Additional file 1: Figure S6). The carriage prevalence remains constant for all age groups (see Additional file 1: Figure S7), indicating that the model has been adequately formulated and calibrated to reproduce the temporal stability of asymptomatic carriage [19]. The model reproduces the endemicity of the IMD, and the age pattern of cases remains roughly constant from year to year, as reported in the literature [63], with fluctuations due only to the demographic shifts (see Additional file 1: Figure S7).
The MCML method estimates both direct and indirect effectiveness
In November 1999, a large mass immunization campaign against MenC disease started in England and Wales, consisting of a routine vaccination of infants and a catchup vaccination targeting people up to 18 years old [52]. The overall MenC IMD incidence fell by about 80 % in the following 2 years. This achievement was accomplished thanks to the high effectiveness of the vaccine, which was found to provide both direct and indirect protection [28].
The likelihood function displays a unique and welldefined maximum in both dimensions, indicating that both the direct effectiveness and indirect effectiveness are identifiable parameters of the model under the conditions investigated and can be coestimated by MCML. The 95 % CI of the most likely values are included in the white dotted line. Fig. 2b, c shows the logarithm of the unidimensional likelihoods of Fig. 2a, as a function of VE_{dir} and VE_{ind}, respectively, obtained by slicing the loglikelihood surface in correspondence of the maximum. The best estimates of VE_{dir} and VE_{ind} and the corresponding 95 % CIs are shown as vertical dashed lines.
Here, we see that the model’s structure allows us to disentangle the two effects of the vaccine, which concur with the global decline of cases. Remarkably, the inferential MCML method can provide an estimate of the two parameters independently and very parsimoniously in terms of field data, comparable with the screening method.
The MCML method accurately estimates MenC VE
To assess the quality of the MCML method, we compared the best estimates of VE_{dir} and VE_{ind} with a number of independent measures of the same quantities reported in the literature.
More precise estimates were published after monitoring the occurrence of MenC disease cases for 10 years [26]. The catchup vaccination was confirmed to be 96 % effective (CI = [92 %, 99 %]). The effectiveness of routine vaccination was slightly higher than previous estimates: 97 % (CI = [91 %, 99 %]).
Considering vaccine uptake statistics and IMD case reports stratified by age and vaccinal status from mid2000 to mid2002, the MCML method estimated the direct effectiveness to be VE_{dir}=96.5 % (red symbols in Fig. 3b), in good agreement with the value found by Campbell et al. [26]. The MCML method reached a higher precision than the screening method (95 % CI = [94.8 %, 97.9 %]), yet required a shorter observation period.
The indirect effectiveness of the MenC vaccine was estimated a few years after the campaign start, based on carriage studies. Figure 3c shows the best estimates and 95 % CIs of the indirect effectiveness, represented as gray dots and whiskers, published in 2002 [28] and 2008 [30]. Comparing the carriage prevalence reported by the 1999 survey and the one observed 1 year later, VE_{ind} was estimated to be 63 % (CI = [ −50 %, 80 %]) [28]. After collecting additional data on the carriage prevalence with a third survey in 2002, VE_{ind} was 75 % (CI = [23 %, 92 %]) [30]. As described in the previous section, the MCML also provided an estimate for VE_{ind}, based on IMD incidence data only. Figure 3c shows in red the best MCML estimate, corresponding to VE_{ind}=68.8 % (95 % CI = [54.4 %, 83.3 %]). Also in this case, the value is in good agreement with those published by Maiden and collaborators and the uncertainty around the point estimate is much smaller.
MCML is more precise than the screening method
Finally, we assessed the applicability and the expected performance of the MCML method in estimating the VE of the 2015 MenB campaign in England. As described in “Methods”, here we assumed that Bexsero can provide only direct protection against IMD.
In Fig. 4a, we assumed VE_{dir}=60 %. The gray area bounded by solid lines indicates the extension of the 95 % confidence limits (CL), calculated using the Farrington formula. The higher 95 % CL converges faster than the lower one to the best estimate. The lower CL requires about 3 years just to reach 0 % effectiveness. To get a fairly precise estimate, we would need to wait longer. If we consider as sufficiently precise a lower 95 % CL not further than 15 % from the point estimate, thus above VE_{dir}=45 %, we must wait 15.6 years from the beginning of the campaign. The shaded red area displays the 95 % CI for VE_{dir}=60 % obtained with the MCML method. Here, the red dots indicate the CLs that we could obtain at mid2016 (9 months of vaccination), mid2017 (1 year and 9 months of vaccination), mid2018, and so on. For all estimates, the CLs of the MCML method are well inside the interval of the screening method. Compared to the screening method, the MCML lower 95 % CLs are about 20fold faster in getting close to the point estimate: after 9 months of observation, the MCML can provide acceptable CIs, with the CLs about 15 % distant from 60 %.
Figure 4b–d show the same comparison between the two methods assuming a best estimate of VE_{dir}=70 %, 80 %, and 90 %, respectively. As expected, as the effectiveness increases, the time required to estimate VE_{dir} precisely becomes shorter for all methods. However, in all panels the red area identifying the CLs of the MCML method always falls well inside the gray area, indicating that the CLs of MCML converge faster than the screening method to the point estimate. As the effectiveness increases to 90 %, the difference between the higher 95 % CLs gets smaller. On the other hand, the lower 95 % CLs remain far enough away for there to be a substantial advantage with the MCML method. For instance, for VE _{best} = 90 % (Fig. 4d), the screening lower 95 % CL reaches 80 % in about 4.6 years. In contrast, the MCML lower 95 % CL reaches 80 % in only 9 months, while in the same 9 months, the screening method is not expected to get over 0 %.
Discussion
The use of MCML methods to infer relevant epidemiological parameters from dynamic epidemic models is becoming more and more important. There are several examples in the literature of similar approaches aimed at characterizing epidemiological quantities that would not be accessible from incidence or prevalence data only, such as the worldwide transmission potential of pandemic flu [64], the environmental contribution to cholera transmission [59], or the interaction between influenza and pneumonia [61]. Here, we have shown that a MCML approach can be useful also in estimating unknown quantities, such as the VE, which could be easily quantified with incidence data as well, but over a longer time frame and with higher uncertainty, given the specific characteristics of meningococcal disease. Overall, our results suggest that a MCML approach to estimate the VE could be generally relevant for vaccines preventing lowincidence diseases, whose effectiveness may need several years to be estimated with enough precision by observational methods. From this perspective, the methodological framework presented in this work is rather general and could be extended in principle to different diseases and immunization scenarios where the limitations of the screening method are known to be relevant [37].
It is important to notice, though, that our approach relies considerably on the integration of a large body of knowledge about the transmission and colonization mechanisms, the emergence of the invasive disease, and the global epidemiology of N. meningitidis. Any extension to different diseases would necessarily require the availability of similar or better knowledge for the pathogen of interest.
 1.
The MCML method fits the whole time series of disease cases before and after vaccination, while estimates by the screening method are based on a single data point in time.
 2.
The dynamic model must be calibrated on epidemiological data by incorporating a large number of assumptions about the population structure, the means of transmission, and the incidence and prevalence of the disease.
While the first element does not carry significant limitations, the calibration on meningococcal epidemiology before the vaccination campaign may affect the MCML results. Specifically, the initial calibration relies on two assumptions: the carriage prevalence across age groups is forced to be stable from year to year and the risk of IMD given transmission by age and per time step is fixed to reproduce the disease endemicity. Despite the abundance of information on meningococcal disease, the relation between disease incidence and carriage prevalence is unclear. In particular, in the literature there are no clinical studies broadly assessing acquisition rates of meningococcal carriage in different age classes. Only a very recent work [65] provided an estimate of acquisition rates, measured by aggregating subjects between 10 and 25 years old. To fill this gap, we had to calculate indirectly the transmission rates β_{ a }, by combining available information on the carriage prevalence by serogroup and the duration of carriage τ_{car}. Different combinations of these parameters may lead to the same transmission rates, but to different MCML estimates of VE. For instance, assuming a shorter duration of carriage is equivalent to considering a smaller fraction of carriage due to a single serogroup.
We tested the sensitivity of our VE estimates on variations of τ_{car}, and we found that estimates of VE_{dir} are fairly unaffected by the assumed value of τ_{car}. In contrast, estimates of VE_{ind} are sensitive to the duration of carriage: assuming a longer duration of carriage would lead to higher estimates of the indirect effectiveness. In Additional file 1: Figure S9, we show and discuss in detail how estimates of VE_{ind} and VE_{dir} for the MenC vaccine vary as τ_{car} is varied between 4 and 8 months, as well as how they evolve in time, with an increasing number of data points available for inference.
In general, there is not a strong consensus around the average duration of asymptomatic carriage, whose assessment requires large longitudinal carriage studies. In a recent review [45], the authors indicated 5–6 months as a reasonable value for τ_{car}, and the most recent modeling efforts rely on this value [43, 62]. We, therefore, chose τ_{car}=6 months as our baseline case. Retrospectively, this appears to be a good choice since it produces the best estimates of VE _{ind} that are in agreement with those that have been published in the literature (Fig. 3). Further applications of our approach would need to identify carefully the most plausible values of τ_{car}; however, it is important to note that changes in τ_{car} do not affect the results presented here for Bexsero, since we only estimate its direct effectiveness. More importantly, additional clinical studies aimed at measuring acquisition rates of meningococcal carriage would be extremely helpful in removing such uncertainties.
We also assumed that the risks of invasive disease given transmission θ_{ a } are constant over time. This assumption, combined with the temporal stability of carriage, reproduces the observed endemicity of meningococcal disease. Therefore, the model does not capture longterm trends in the incidence of disease, such as the decline of invasive disease observed in MenB or the rise of MenC cases observed just before vaccination. The mechanisms behind such trends are not completely understood so we do not try to model them; however, this assumption could lead to over or underestimating the VE. To address this point, we tested whether, in the MenC setting, the inference of VE is affected when the model is calibrated assuming different values of θ, that is, different values of the total number of IMD cases occurring every year. As reported in Additional file 1, we found that estimates of the direct effectiveness are robust against variations in the assumed number of IMD cases generated by the model. Even by varying the number of expected IMD cases in 1999 from 600 to 1000, against a baseline value of 869, the MCML estimate of VE_{dir} is basically unaffected. On the other hand, estimates of VE_{ind} may vary significantly, as reported in Additional file 1: Table S4.
In summary, the MCML method is based on stochastic simulations of a dynamical compartmental model that describes the transmission of N. meningitidis in the population. The model is calibrated on epidemic data before vaccination, and, once the model is calibrated, the parameters VE_{dir} and VE_{ind} are uniquely identified via MCML. We carefully considered the impact of all the assumptions we made to parameterize the model fully before vaccination and found that, overall, estimates of VE_{dir} are very robust to changes in the initial calibration assumptions, thus supporting the validity of our method as a good candidate for an initial assessment of VE from field data. On the other hand, we found that estimates of VE_{ind} are significantly affected by the specific choice of parameterization, which, therefore, should be carefully assessed before evaluating the indirect effect of a meningococcal vaccine.
Finally, we tested the robustness of the MCML estimates against changes in the duration of vaccineinduced protection τ_{imm} for the routine vaccination in the MenC campaign. By varying the duration of immunity between 9 months and 24 months, we found that estimates of both VE_{dir} and VE_{ind} are not significantly affected by this parameter, as shown in Additional file 1: Figure S10.
Another important assumption we made is that a single strain model can be used to describe the transmission dynamics of different meningococcal serogroups (C and B), thus neglecting the interaction between serogroups. Although modeling this interaction could explain more realistically the agedependent risk of disease given infection [42], and it might impact our estimates of VE [66], it comes at the cost of introducing several unknown parameters that must be estimated with the little empirical knowledge available. Here, we opted for a more parsimonious approach in terms of parameterization aiming at reducing all possible model uncertainties, thus minimizing the uncertainty on the VE. Moreover, the interaction between serogroups is expected to play a relevant role on long time scales, such as decades or hundreds of years [66], while we are interested only in the dynamics of the first few years after vaccine introduction.
Conclusions
In this work, we have shown how a MCML method, combining stochastic simulations of a dynamic model and field data, can provide a fast and accurate estimate of the effectiveness of vaccines against meningococcal disease, by integrating epidemiological and demographic knowledge into an inferential framework.
We have retrospectively tested the MCML method on the MenC vaccination campaign that started in 1999 in England and found estimates of the VE that are in good agreement with those obtained with the classic screening method. In this context, we showed three main advantages of the MCML method: the shorter time required to obtain estimates, the higher precision in terms of CIs, and the ability to quantify both the direct and indirect effect of the vaccine, based on disease incidence data only.
Finally, we have shown how these advantages could have a high practical importance in estimating the effectiveness of Bexsero, a multicomponent vaccine that is currently being administered in a mass immunization campaign that started in England in September 2015. It will be important to test the MCML method for the ongoing campaign as soon as data on IMD incidence are available. This approach could provide an initial estimate of the VE, which, along with traditional observational methods, could support the work of public health officials. Furthermore, Bexsero is made with four antigenic components. The method presented in this work could be extended by considering the four antigens as individual strains to assess how each component contributes to the overall VE.
Abbreviations
C, carrier not vaccinated; CI, confidence interval; CL, confidence limit; CV, carrier vaccinated but not immune; CVI, carrier vaccinated and immune; IMD, invasive meningococcal disease; MCML, Monte Carlo maximum likelihood; MenB, serogroup B meningococcal; MenC, serogroup C meningococcal; PHE, Public Health England; S, susceptible not vaccinated; SIS, SusceptibleInfectedSusceptible; SMC, sequential Monte Carlo; SV, susceptible vaccinated but not immune; SVI, susceptible vaccinated and immune; VE, vaccine effectiveness
Declarations
Acknowledgements
We gratefully acknowledge Helen Campbell for useful comments and for providing the incidence data of IMD in England. We are grateful to Mirna Roberts and Nick Andrews for useful comments and discussions in the early stage of this work. We thank Riccardo Beltrami and Nicola Pacchiani for their support with the computational infrastructure of the GSK Scientific Computing Lab.
Funding
LA received a PhD fellowship from GSK. LA and MT received support from the Lagrange Project of the ISI Foundation funded by the CRT Foundation.
Availability of data and materials
The population distribution of England is available as Additional file 2. The distribution of meningococcal carriage by age taken from [19] is available as Additional file 3. Time series of IMD in England by serogroup are available upon request directed to the Immunisation, Hepatitis and Blood Safety Department, National Infection Service, PHE.
Authors’ contributions
LA, MT, and DM conceived and designed the study. LA collected data; designed, implemented, and ran the computational model; and analyzed the simulation results. LA, MT, and DM wrote the manuscript. All authors read and approved the final version of the manuscript.
Competing interests
LA has received a PhD fellowship funded by GSK Vaccines. DM is a fulltime employee of GSK Vaccines.
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
Authors’ Affiliations
References
 World Health Organization. Meningococcal vaccines: polysaccharide and polysaccharide conjugate vaccines. Wkly Epidemiol Rec. 2002; 77(40):331–9.Google Scholar
 Harrison OB, Claus H, Jiang Y, Bennett JS, Bratcher HB, Jolley KA, et al.Description and nomenclature of Neisseria meningitidis capsule locus. Emerg Infect Dis. 2013; 19(4):566–73. doi:http://0dx.doi.org.brum.beds.ac.uk/10.3201/eid1904.111799.View ArticlePubMedPubMed CentralGoogle Scholar
 Public Health Agency of Canada. Enhanced surveillance of invasive meningococcal disease in Canada: 1 January, 2004, through 31 December, 2005. Can Commun Dis Rep. 2007; 33(1481–8531 (Electronic)):1–15.Google Scholar
 Ines Agudelo C, Sanabria OM, Ovalle MV. Serogroup Y meningococcal disease, Colombia. Emerg Infect Dis. 2008; 14(6):990–1. doi:http://0dx.doi.org.brum.beds.ac.uk/10.3201/eid1406.071357.View ArticlePubMedGoogle Scholar
 Ciccone FH, Suzuki E, Pellini A. Meningococcal disease: communitarian outbreak investigation in Grajau, in the city of Sao Paulo. BEPA. 2006; 3:7–12.Google Scholar
 Chiavetta L, Chavez E, Ruzic A, Mollerach M, Regueira M. Surveillance of Neisseria meningitidis in Argentina, 1993–2005: distribution of serogroups, serotypes and serosubtypes isolated from invasive disease. Rev Argent Microbiol. 2007; 39(1):21–7.PubMedGoogle Scholar
 Nicolas P, Norheim G, Garnotel E, Djibo S, Caugant DA. Molecular epidemiology of Neisseria meningitidis isolated in the African Meningitis Belt between 1988 and 2003 shows dominance of sequence type 5 (ST5) and ST11 complexes. J Clin Microbiol. 2005; 43(10):5129–35. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1128/JCM.43.10.51295135.2005.View ArticlePubMedPubMed CentralGoogle Scholar
 Coulson GB, von Gottberg A, du Plessis M, Smith AM, de Gouveia L, Klugman KP. Meningococcal disease in South Africa, 1999–2002. Emerg Infect Dis. 2007; 13(2):273–81. doi:http://0dx.doi.org.brum.beds.ac.uk/10.3201/eid1302.051553.View ArticlePubMedPubMed CentralGoogle Scholar
 Chiou CS, Liao JC, Liao TL, Li CC, Chou CY, Chang HL, et al.Molecular epidemiology and emergence of worldwide epidemic clones of Neisseria meningitidis in Taiwan. BMC Infect Dis. 2006; 6:25. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1186/14712334625.View ArticlePubMedPubMed CentralGoogle Scholar
 Takahashi H, Kuroki T, Watanabe Y, Tanaka H, Inouye H, Yamai S, et al.Characterization of Neisseria meningitidis isolates collected from 1974 to 2003 in Japan by multilocus sequence typing. J Med Microbiol. 2004; 53(0022–2615):657–62. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1099/jmm.0.455410.View ArticlePubMedGoogle Scholar
 Australian Gonococcal Surveillance Programme, Lara N. Annual report of the Australian gonococcal surveillance programme, 2014. Commun Dis Intell Q Rep. 2015; 39(3):347–54.Google Scholar
 Black S, Pizza M, Nissum M, Rappuoli R. Toward a meningitisfree world. Sci Trans Med. 2012; 4(123):123–5. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1126/scitranslmed.3003859.View ArticleGoogle Scholar
 Centers for Disease Control and Prevention. Meningococcal disease In: Hamborsky J, Kroger A, Wolfe S, editors. Epidemiology and prevention of vaccinepreventable diseases. Vol. 39. 13th ed: 1993. p. 3–25. Chap. 14. http://www.cdc.gov/vaccines/pubs/pinkbook/downloads/mening.pdf.
 Thompson MJ, Ninis N, Perera R, MayonWhite R, Phillips C, Bailey L, et al.Clinical recognition of meningococcal disease in children and adolescents. Lancet. 2006; 367(9508):397–403. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1016/S01406736(06)679324.View ArticlePubMedGoogle Scholar
 Rosenstein NE, Perkins BA, Stephens DS, Popovic T, Hughes JM. Meningococcal disease. N Engl J Med. 2001; 344(18):1378–88. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1056/NEJM200105033441807.View ArticlePubMedGoogle Scholar
 European Centre for Disease Prevention and Control. Surveillance of invasive bacterial diseases in Europe 2012. 2015:89. doi:http://0dx.doi.org.brum.beds.ac.uk/10.2900/1510. http://www.worldcat.org/title/surveillanceofinvasivebacterialdiseasesineurope20082009/oclc/759934800.
 WHO. Meningococcal meningitis fact sheet. 2015. http://www.who.int/mediacentre/factsheets/fs141/en/. Accessed 11 Nov 2015.
 Trotter CL, Gay NJ, Edmunds WJ. The natural history of meningococcal carriage and disease. Epidemiol Infect. 2006; 134(3):556–66. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1017/S0950268805005339.View ArticlePubMedGoogle Scholar
 Christensen H, May M, Bowen L, Hickman M, Trotter CL. Meningococcal carriage by age: a systematic review and metaanalysis. Lancet Infect Dis. 2010; 10(12):853–61. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1016/S14733099(10)702516.View ArticlePubMedGoogle Scholar
 Tan LKK, Carlone GM, Borrow R. Advances in the development of vaccines against Neisseria meningitidis. N Engl J Med. 2010; 362(16):1511–20. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1016/S00843873(10)79653X.View ArticlePubMedGoogle Scholar
 Ramsay ME, Andrews N, Kaczmarski EB, Miller E. Efficacy of meningococcal serogroup C conjugate vaccine in teenagers and toddlers in England. Lancet. 2001; 357(9251):195–6. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1016/S01406736(00)035947.View ArticlePubMedGoogle Scholar
 Costantino P, Viti S, Podda A, Velmonte MA, Nencioni L, Rappuoli R. Development and phase 1 clinical testing of a conjugate vaccine against meningococcus A and C. Vaccine. 1992; 10(10):691–8. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1016/0264410X(92)90091W.View ArticlePubMedGoogle Scholar
 Snape MD, Pollard AJ. Meningococcal polysaccharide–protein conjugate vaccines. Lancet Infect Dis. 2005; 5(1):21–30. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1016/S14733099(04)012514.View ArticlePubMedGoogle Scholar
 Bose A, Coen P, Tully J, Viner R, Booy R. Effectiveness of meningococcal C conjugate vaccine in teenagers in England. Lancet. 2003; 361(9358):675–6. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1016/S01406736(03)125639.View ArticlePubMedGoogle Scholar
 Trotter CL, Andrews NJ, Kaczmarski EB, Miller E, Ramsay ME. Effectiveness of meningococcal serogroup C conjugate vaccine 4 years after introduction. Lancet. 2004; 364(9431):365–7. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1016/S01406736(04)167251.View ArticlePubMedGoogle Scholar
 Campbell H, Andrews N, Borrow R, Trotter C, Miller E. Updated postlicensure surveillance of the meningococcal C conjugate vaccine in England and Wales: effectiveness, validation of serological correlates of protection, and modeling predictions of the duration of herd immunity. Clin Vaccine Immunol. 2010; 17(5):840–7. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1128/CVI.0052909.View ArticlePubMedPubMed CentralGoogle Scholar
 Borrow R, Abad R, Trotter C, van der Klis FRM, Vazquez JA. Effectiveness of meningococcal serogroup C vaccine programmes. Vaccine. 2013; 31(41):4477–86. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1016/j.vaccine.2013.07.083.View ArticlePubMedGoogle Scholar
 Maiden MC, Stuart JM. Carriage of serogroup C meningococci 1 year after meningococcal C conjugate polysaccharide vaccination. Lancet. 2002; 359(9320):1829–30. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1016/S01406736(02)086798.View ArticlePubMedGoogle Scholar
 Ramsay ME. Herd immunity from meningococcal serogroup C conjugate vaccination in England: database analysis. BMJ. 2003; 326(7385):365–6. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1136/bmj.326.7385.365.View ArticlePubMedPubMed CentralGoogle Scholar
 Maiden MCJ, IbarzPavón AB, Urwin R, Gray SJ, Andrews NJ, Clarke SC, et al.Impact of meningococcal serogroup C conjugate vaccines on carriage and herd immunity. J Infect Dis. 2008; 197(5):737–43. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1086/527401.View ArticlePubMedGoogle Scholar
 Vernikos G, Medini D. Bexsero®; chronicle. Pathogens Global Health. 2014; 108(7):305–16. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1179/2047773214Y.0000000162.View ArticlePubMedPubMed CentralGoogle Scholar
 Donnelly J, Medini D, Boccadifuoco G, Biolchi A, Ward J, Frasch C, et al.Qualitative and quantitative assessment of meningococcal antigens to evaluate the potential strain coverage of proteinbased vaccines. Proc Nat Acad Sci. 2010; 107(45):19490–5. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1073/pnas.1013758107.View ArticlePubMedPubMed CentralGoogle Scholar
 Vogel U, Taha MK, Vazquez JA, Findlow J, Claus H, Stefanelli P, et al.Predicted strain coverage of a meningococcal multicomponent vaccine (4CMenB) in Europe: a qualitative and quantitative assessment. Lancet Infect Dis. 2013; 13(5):416–25. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1016/S14733099(13)700069.View ArticlePubMedGoogle Scholar
 Halloran ME, Longini Jr IM, Struchiner CJ. Design and analysis of vaccine studies: introduction. SpringerVerlag New York: Stat Biol Health: 2009. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1007/9780387686363.
 Farrington CP. Estimation of vaccine effectiveness using the screening method. Int J Epidemiol. 1993; 22(4):742–6. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1093/ije/22.4.742.View ArticlePubMedGoogle Scholar
 Orenstein WA, Bernier RH, Dondero TJ, Hinman AR, Marks JS, Bart KJ, et al.Field evaluation of vaccine efficacy. Bull World Health Org. 1985; 63(6):1055–68.PubMedPubMed CentralGoogle Scholar
 Cohen AL, Taylor T, Farley MM, Schaffner W, Lesher LJ, Gershman KA, et al.An assessment of the screening method to evaluate vaccine effectiveness: the case of 7valent pneumococcal conjugate vaccine in the United States. PloS One. 2012; 7(8):41785. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1371/journal.pone.0041785.View ArticleGoogle Scholar
 Minodier L, Blanchon T, Souty C, Turbelin C, Leccia F, Varesi L, et al.Influenza vaccine effectiveness: best practice and current limitations of the screening method and their implications for the clinic. Expert Rev Vacc. 2014; 13(8):1039–48. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1586/14760584.2014.930666.View ArticleGoogle Scholar
 Public Health England. Meningococcal disease: laboratory confirmed cases in England and Wales. https://www.gov.uk/government/publications. Accessed 11 Nov 2015.
 Trotter CL, Edmunds WJ. Reassessing the costeffectiveness of meningococcal serogroup C conjugate (MCC) vaccines using a transmission dynamic model. Med Decis Mak: Int J Soc Med Decis Mak. 2006; 26(1):38–47. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1177/0272989X05284109.View ArticleGoogle Scholar
 Trotter CL. Dynamic models of meningococcal carriage, disease, and the impact of serogroup C conjugate vaccination. Am J Epidemiol. 2005; 162(1):89–100. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1093/aje/kwi160.View ArticlePubMedGoogle Scholar
 Guzzetta G, Manfredi P, Gasparini R, Panatto D, Edmunds WJ. On the relationship between meningococcal transmission dynamics and disease: remarks on humoral immunity. Vaccine. 2009; 27(25–26):3429–34. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1016/j.vaccine.2009.01.092.View ArticlePubMedGoogle Scholar
 Christensen H, Hickman M, Edmunds WJ, Trotter CL. Introducing vaccination against serogroup B meningococcal disease: an economic and mathematical modelling study of potential impact. Vaccine. 2013; 31(23):2638–46. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1016/j.vaccine.2013.03.034.View ArticlePubMedPubMed CentralGoogle Scholar
 Arulampalam MS, Maskell S, Gordon N, Clapp T. A tutorial on particle filters for online nonlinear/nonGaussian Bayesian tracking. IEEE Trans Signal Process. 2002; 50(2):174–88. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1109/78.978374.View ArticleGoogle Scholar
 Caugant DA, Maiden MCJ. Meningococcal carriage and disease – population biology and evolution. Vaccine. 2009; 27:64–70. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1016/j.vaccine.2009.04.061.View ArticleGoogle Scholar
 Øvstebø R, Hellerud BC, Coureuil M, Nassif X, Brandtzaeg P. Pathogenesis of invasive disease In: Feavers I, Pollard AJ, Sadarangani M, editors. Handbook of meningococcal disease management. Switzerland: Springer International Publishing: 2016. p. 25–43. Chap. 3. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1007/9783319281193.Google Scholar
 Office of National Statistics. Population. http://ons.gov.uk/ons/taxonomy/index.html?nscl=Population. Accessed 11 Nov 2015.
 Mossong J, Hens N, Jit M, Beutels P, Auranen K, Mikolajczyk R, et al.Social contacts and mixing patterns relevant to the spread of infectious diseases. PLoS Med. 2008; 5(3):74. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1371/journal.pmed.0050074.View ArticleGoogle Scholar
 De Wals P, Bouckaert A. Methods for estimating the duration of bacterial carriage. Int J Epidemiol. 1985; 14(4):628–34. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1093/ije/14.4.628.View ArticlePubMedGoogle Scholar
 Caugant DA, Fogg C, Bajunirwe F, Piola P, Twesigye R, Mutebi F, et al.Pharyngeal carriage of Neisseria meningitidis in 2–19yearold individuals in Uganda. Trans R Soc Trop Med Hyg. 2006; 100(12):1159–63. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1016/j.trstmh.2006.01.004.View ArticlePubMedGoogle Scholar
 Glitza IC, Ehrhard I, MüllerPebody B, Reintjes R, Breuer T, Ammon A, et al.Longitudinal study of meningococcal carrier rates in teenagers. Int J Hygiene Environ Health. 2008; 211(3–4):263–72. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1016/j.ijheh.2007.05.006.View ArticleGoogle Scholar
 Trotter CL, Ramsay ME, Kaczmarski EB. Meningococcal serogroup C conjugate vaccination in England and Wales: coverage and initial impact of the campaign. Commun Dis Public Health/PHLS. 2002; 5(3):220–5.Google Scholar
 Wise J. Meningitis B vaccine to be introduced in UK after U turn on its cost effectiveness. BMJ (Clinical research ed.) 2014; 348(March):2327. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1136/bmj.g2327.Google Scholar
 Public Health England. National childhood immunisation programme boosted by MenB vaccine. 2015. https://www.gov.uk/government/news/nationalchildhoodimmunisationprogrammeboostedbymenbvaccine. Accessed 11 Nov 2015.
 Andrews SM, Pollard AJ. A vaccine against serogroup B Neisseria meningitidis: dealing with uncertainty. Lancet Infect Dis. 2014; 14(5):426–34. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1016/S14733099(13)703414.View ArticlePubMedGoogle Scholar
 Abio A, Neal KR, Beck CR. An epidemiological review of changes in meningococcal biology during the last 100 years. Pathogens Global Health. 2013; 107(7):373–80. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1179/2047773213Y.0000000119.View ArticlePubMedPubMed CentralGoogle Scholar
 Trotter C, Ramsay M, Harrison L. Introduction and epidemiology of meningococcal disease In: Feavers I, Pollard AJ, Sadarangani M, editors. Handbook of meningococcal disease management. Switzerland: Springer International Publishing: 2016. p. 1–14. Chap. 1. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1007/9783319281193.Google Scholar
 Ionides EL, Breto C, King AA. Inference for nonlinear dynamical systems. Proc Nat Acad Sci. 2006; 103(49):18438–43. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1073/pnas.0603181103.View ArticlePubMedPubMed CentralGoogle Scholar
 King AA, Ionides EL, Pascual M, Bouma MJ. Inapparent infections and cholera dynamics. Nature. 2008; 454(7206):877–80. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1038/nature07084.View ArticlePubMedGoogle Scholar
 Camacho A, Ballesteros S, Graham AL, Carrat F, Ratmann O, Cazelles B. Explaining rapid reinfections in multiplewave influenza outbreaks: Tristan da Cunha 1971 epidemic as a case study. Proc R Soc B: Biol Sci. 2011; 278(1725):3635–43. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1098/rspb.2011.0300.View ArticleGoogle Scholar
 Shrestha S, Foxman B, Weinberger DM, Steiner C, Viboud C, Rohani P. Identifying the interaction between influenza and pneumococcal pneumonia using incidence data. Sci Trans Med. 2013; 5(191):84. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1126/scitranslmed.3005982.View ArticleGoogle Scholar
 Christensen H, Trotter CL, Hickman M, Edmunds WJ. Reevaluating cost effectiveness of universal meningitis vaccination (Bexsero) in England: modelling study. BMJ. 2014; 349:5725. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1136/bmj.g5725.View ArticleGoogle Scholar
 Gray SJ, Trotter CL, Ramsay ME, Guiver M, Fox AJ, Borrow R, et al.Epidemiology of meningococcal disease in England and Wales 1993/94 to 2003/04: contribution and experiences of the Meningococcal Reference Unit. J Med Microbiol. 2006; 55(Pt 7):887–96. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1099/jmm.0.462880.View ArticlePubMedGoogle Scholar
 Tizzoni M, Bajardi P, Poletto C, Ramasco JJ, Balcan D, Gonçalves B, et al.Realtime numerical forecast of global epidemic spreading: case study of 2009 A/H1N1pdm. BMC Med. 2012; 10(1):165. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1186/1741701510165.View ArticlePubMedPubMed CentralGoogle Scholar
 Jeppesen CA, Snape MD, Robinson H, Gossger N, John TM, Voysey M, et al.Meningococcal carriage in adolescents in the United Kingdom to inform timing of an adolescent vaccination strategy. J Infect. 2015; 71(1):43–52. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1016/j.jinf.2015.02.006.View ArticlePubMedGoogle Scholar
 Poore KD, Bauch CT. The impact of aggregating serogroups in dynamic models of Neisseria meningitidis transmission. BMC Infect Dis. 2015; 15:300. doi:http://0dx.doi.org.brum.beds.ac.uk/10.1186/s1287901510158.View ArticlePubMedPubMed CentralGoogle Scholar