1. Home
  2. Blog
  3. A plea to Tobacco Control - do not disregard this important research

A plea to Tobacco Control - do not disregard this important research

This week saw the release of an important study by Dr. Riccardo Polosa and colleagues, which followed 300 people using electronic cigarettes over the period of a year. I won't write up the study, as it's been done elsewhere very well.

What I do want to focus a little attention on is an early indication that the study might be being dismissed by those senior in Tobacco Control (TC). To some readers, this might come across as an understatement, but I want to use this piece to try to persuade those involved in TC that they must give credit to the study, and to address some early concerns that have been flagged.

I was alerted to resistance to the study by a tweet from Roberto Bertolini:

When pressed on this, here is what he had to say:


And that was it, until a few days later when Greg Conley from CASAA tweeted to ask whether anyone in the Public Health sphere had engaged with this study. I tweeted back to say that Roberto Bertollini had, but then failed to follow up on his claims. Dr. Bertollini responded to the conversation, stating that one cannot explain tricky methodological questions in 140 characters. To be fair, this is quite true, and he did state that he would follow up in the appropriate venue - he has been offered two venues so far, and I hope he will engage. I also offer this space to him to engage, if he so desires. 

I think it reasonable that I first take a few minutes to set the case as to why I think this is an important piece of research, and to anticipate a couple of his other methodological complaints. Dr. Bertollini claims two specific methodological issues in his tweet: Lost to follow-up (40%) and blindness to treatment after week 12. Regarding the first, this is true enough, 40% of cases were indeed lost to follow up. In other words, out of the 300 that joined the study, 40% simply did not complete the study, and did not respond.

Intention to treat

This is not an unusual figure for longitudinal research. There are ways of ensuring that participants are not lost, but these are often uneconomical especially in the context of longitudinal research, which is exceptionally expensive to conduct in the first place. Instead, the gold standard approach is to conduct an 'intention to treat' analysis on the data from the full sample. What this means is that the 40% who drop out of the study are counted as failures, and this is exactly what was done in this study.

In other words, then, the 40% drop out that Bertollini refers to actually bolsters the stated results, as it is possible that this sample contains some number (however small) of people that would in fact have confirmed the hypothesis: that e-cigarettes help users to quit. In other words, then, I'm not sure why Dr. Bertollini views this as a methodological failing, at least not when it comes to the authors conclusions. 

Indeed, as Michael Siegel has pointed out recently, a study published in Tobacco Control journal last month entirely failed to conduct such an intention to treat analysis, resulting in a hugely inflated 18% quit rate after 6 months. In fact, had the authors of this study used the same methodology as Polosa et al., and included the 58% lost to follow up, the intent to treat analysis would have been reported as 9% - this seems to me to be fairly scandalous, and does serious damage to the credibility of the journal in which it was published: why no outcry on this piece of work?

The sample

Dr. Polosa recruited his sample via newspapers, asking for smokers who had no intention to quit smoking to take part in the study. "No intention to quit" is a slippery concept, as having a conscious motivation to quit could be argued to be dissociable from having a pre-conscious (or nonconscious) desire to do so. Nevertheless, despite the validity (or lack of) this sample's specific characteristics, they were still all smokers. Furthermore, "no intention to quit" is still a label that these participants had applied to themselves, so to this extent it's a valid sample.


I say "confirm the hypothesis" above, but there actually wasn't one explicitly stated- and this is one of the critiques Dr. Bertollini might point to. It is standard to state a hypothesis prior to testing, for a number of reasons relating to probability. Specifically, the nature of the hypothesis (1-tailed or 2-tailed) will define the value by which you measure statistical significance once the data comes in. There are, in fact, three hypotheses implicit in the study (and these are all 2-tailed, since the direction of the expected result is not stated). The authors address this specific point by stating the exploratory nature of their investigation in the introduction, and also in their statistical analyses section. Another standard methodological tool "missing" in this study is the placebo control.

Lack of placebo

Dr. Farsalinos has explained why it is not possible to have a placebo electronic cigarette. To wit: e-cigs don't just deliver nicotine, but a whole range of sensory aspects which are critical to their acceptance by smokers. All vapers know this to be true - throat hit, plumes of vapor, tactile sensations, even the crackle and pop of the juice as it is heated by the atomizer. All this is key to the e-cigarette experience, and it is not clear that it is possible to create a placebo version of this which would be a useful control. Instead of a placebo, the authors chose to simply compare outcomes with three strengths of nicotine, and I think this is entirely reasonable.

Furthermore, an interesting point to note is that the time will soon pass in which it is even possible to find a reliable sample of smokers who have never used an electronic cigarette. This study began in 2009, when e-cigarettes were more-or-less unheard of. Vapers learn very quickly to differentiate between nicotine strengths, and I have no idea how you could possibly hope to create a similar sensory experience between 3 strengths of nicotine. Add capsaicin? But then you're introducing big new confounding variables.

Statistical analysis

A conservative case for statistical analysis might claim that since 3 hypotheses are being tested in the one sample, there chance of a type 1 error is increased and the significance threshold should, therefore, be corrected. This is a conservative case, and arguments against it would relate to the independence of the hypotheses (I suggest that they are fully independent), and that correcting might unacceptably raise the chance of a type 2 error. In reality, this study is exploratory and there is no doubt that replication is needed in the future, which brings me to my final point:

The final methodological "failing" that needs to be flagged up is best illustrated by this photograph:

Categoria e-cigarette

In case you're not familiar with the currently available electronic cigarettes, the picture above is from a 1st generation electronic cigarette that is as far removed from today's vaping experience as a carphone is from a smart phone.

This is important, and it also represents a further big challenge for people researching electronic cigarettes: the products available today to vapers are as varied as any product on the market. How do you do an ecologically valid study, considering the variety of devices people are actually encountering and using? The sheer number of devices, accessories, liquids and so forth is a function of the serious and rapid development this industry has undertaken in the last 5 years. And this development has taken place because vapers have had the ability to directly communicate their needs to manufacturers, who have then responded very, very quickly.

In our (ECF) own research, we have found that vapers are hugely varied in their needs, and no single product available on the market today works for everyone. This is the central reason why, for many, the medicinalization of electronic cigarettes will result in a de-facto ban for many users, and why it is such poor public health policy.

Final points

This study is not without its limitations. I don't think any serious researcher would disagree with that, but to claim that the conclusions are unfounded is not true. Dr. Polosa has spoken candidly about these limitations, and the need to conduct further experiments, but it is the first piece of research into the specific question of the efficacy of e-cigarettes for those not intending to quit.

On ECF, we hear time and time again of smokers who tried an e-cigarette motivated not by any intention to quit, but to save money, or to have access to nicotine where cigarettes are prohibited. And the story is often the same: these self same smokers found that, over time, they found they preferred e-cigarettes and eventually quit, despite not setting out with any original motivation to do so. To me, the research Dr. Polosa and his colleagues have embarked on reflects this grouping.

I believe that with rigorous methodology, and newer equipment we will see far higher success rates and I encourage Tobacco Control scientists to join in with this process and engage with people like Dr. Polosa and Dr. Farsalinos who see real potential offered by electronic cigarettes in offering smokers a third way out NOW.

I will give the final word to Dr. Farsalinos, because I think this sums up the study very succinctly:

Overall, this is an important study. Results are not impressive, but that was expected when you use an old device which is inefficient (and is not currently produced). The major conclusion is that the potential of e-cigarettes to reduce or completely substitute smoking is a reality. What we already know from experience has been proven by a clinical trial. Studies should now be focused more on safety.


blog comments powered by Disqus