Science, right. If you need help with the big words Kaiser just ask.
First the ahem, study.
http://www.organic-systems.org/journal/81/8106.pdf
Anyway, To tear it apart Mark Lynas does a good job, saves me a bunch of
typing but came to the same conclusion:
GMO pigs study – more junk science
When I saw on Twitter that a ‘major new peer-reviewed study’ was about
to reveal serious health impacts from GMO corn and soya, I was intrigued
to say the least. Would this be Seralini 2.0, a propaganda effort by
anti-biotech campaigners masquerading as proper science, or something
truly new and ground-breaking?
Extraordinary claims require extraordinary evidence – and it would take
a lot of extraordinary evidence to confound the hundreds of studies
showing that GMO foods are just as safe as conventional, as summarised
in this recent AAAS statement:
“The science is quite clear: crop improvement by the modern molecular
techniques of biotechnology is safe. The World Health Organization, the
American Medical Association, the U.S. National Academy of Sciences, the
British Royal Society, and every other respected organization that has
examined the evidence has come to the same conclusion: consuming foods
containing ingredients derived from GM crops is no riskier than
consuming the same foods containing ingredients from crop plants
modified by conventional plant improvement techniques.”
So when I found the paper, again via Twitter, I determined to read it as
I would a climate ‘denier’ paper which aimed to overturn the scientific
consensus in that area – with an open mind, but a sceptical one. I could
see that it was already generating news, and the anti-GMO crowd on
Twitter were also getting excited about some new grist to their
ideological mill. Here’s what Reuters wrote:
“Pigs fed a diet of only genetically modified grain showed markedly
higher stomach inflammation than pigs who dined on conventional feed,
according to a new study by a team of Australian scientists and U.S.
researchers.”
Really? Time to have a look at the study. It is by a Judy Carman and
colleagues, entitled ‘A long-term toxicology study on pigs fed a
combined genetically modified (GM) soy and GM maize diet’ and published
in a minor Australian journal I have never heard of called ‘Journal of
Organic Systems’. (This journal does not appear in PubMed, suggesting it
is not taken very seriously in the scientific community. It only
publishes about twice a year, mostly with research touting the benefits
of organic agriculture.)
I skimmed the paper first, and the conclusions seemed doubtful enough
(see below) to try to find out who was beind it. So I looked at the
sponsors of this journal. They include the Organic Federation of
Australia, which seemed odd for a journal presumably aiming to be
independent. Imagine the hullaballoo if Nature Biotechnology was
sponsored by Monsanto!
I also wondered who Judy Carman and her colleagues were. She turns out
to be – as I feared – a long-time anti-biotech campaigner, with a
website called ‘GMOJudyCarman‘, which says it is supported by
GMOSeralini.org. She is a founding member, according to this website, of
the scientific advisory council of the Sustainable Food Trust. The
Sustainable Food Trust was the UK outfit set up by former organic
lobbyist Patrick Holden which stage-managed the media release of the
infamous Seralini GMO rats study to the Daily Mail and other credulous
outlets.
What about the co-authors? One is a Howard Vlieger, who seems to have
made some wild allegations about GMOs in the past if this source is to
be believed. Vlieger is president and co-founder of Verity Farms, a US
‘natural foods’ outfit which markets non-GMO grain. Despite this, the
paper declares that the authors have no conflicts of interest, although
it seems to me that he would have a very clear commercial interest in
scaring people about GMOs in order to drum up business of his GMO-free
offerings.
What about funding? The paper states that funding came from Verity
Farms, the natural product outfit mentioned above. Carman and her
colleagues are also funded by and associated with the Institute of
Health and Environmental Research, an Australian not-for-profit which
seems to be entirely dedicated to anti-GMO activism. Recent activities
have included opposing Bt brinjal in India and CSIRO’s GMO wheat in
Australia. Funding sources are not disclosed, although donations are
solicited. The paper’s acknowledgements are a veritable who’s who of
anti-biotech activism, includin Jeffrey Smith, John Fagan and Arpad
Pusztai.
So, that’s the context. Now let’s look at what raised my suspicions
about the actual study. Well, Carman and colleagues claim significant
differences in a long-term study of pigs fed GMO and non-GMO diets. But
if you look at the data they present (and the data presentation is at
least a step better than Seralini) there are obvious problems. Clearly
all the animals were in very poor health – weaner mortality is reported
as 13% and 14% in GM-fed and non-GM fed groups, which they claim is
“within expected rates for US commercial piggeries”, a vague statement
intended to justify what seem to have been inadequate husbandry
standards.
This picture is even more stark in the data presented in Table 3. 15% of
non-GM fed pigs had heart abnormalities, while only 6% of GM-fed pigs
did so. Similarly, twice as many non-GM pigs as GM ones had liver
problems. Why no headlines here? “Pigs fed non-GMO feed 100% more likely
to develop heart and liver problems, study finds” – I can just see it in
the Daily Mail. But of course negative results were not what Carman et
al were looking for.
So we fast-forward to the stomach inflammations. This is where Carman et
al got their headline. As Reuters reported:
“But those pigs that ate the GM diet had a higher rate of severe stomach
inflammation – 32 percent of GM-fed pigs compared to 12 percent of non-
GM-fed pigs. The inflammation was worse in GM-fed males compared to non-
GM fed males by a factor of 4.0, and GM-fed females compared to non-GM-
fed females by a factor of 2.2.”
This is statistical fishing of the most egregious sort, and I would put
money on the Reuters summary above being lifted near-verbatim from a
press release written by Carman et al. Table 3 actually shows that many
more pigs fed non-GMO feed had stomach inflammations than those with GMO
feed. So 31 non-GM pigs had ‘mild’ inflammation, while only 23 GM pigs
had it. For ‘moderate’ inflammation, a GMO diet again seemed to be
beneficial: 29 non-GM pigs had moderate inflammation of the stomach,
while 18 had it. So that’s 40% vs 25%. Do Carman et al perform a test
for statistical significance to see if GMO feed has a protective effect
on pigs stomachs? Of course not – that’s not the result they are after.
These findings are ignored.
Instead, it is the next line of data that they play up: for ‘severe’
inflammation 9 non-GM pigs were determined to have it, while 23 GM-fed
pigs had it. Shock, horror. You can immediately see how the data is all
over the place from the previous results, which also rule out any causal
mechanism with GMO feed – if GMO feed is causing the severe
inflammation, why is the non-GMO feed causing far more mild to moderate
inflammation? It’s clearly just chance, and all the pigs are not doing
well and suffering stomach problems: about 60% of both sets had stomach
erosion.
Yet the paper slyly presents photographs of inflamed pigs stomachs, with
non-inflamed and mildly inflamed from non-GM fed pigs, and moderate and
severe inflammation presented from GM-fed pigs. Yet 38 of the non-GM
pigs, more than half of the total of 73, were suffering moderate or
severe inflammation – why not present photos of their stomachs? This is
rather reminiscent of how Seralini presented shocking pictures of GM-fed
rats with massive cancerous tumours, but did not present pictures of the
control rats (non-GM fed) which also developed cancers.
Indeed, if you add together the ‘moderate’ and ‘extreme’ categories –
which from the photos are not easy to tell apart, involving a value
judgement on the part of the vets employed to do the post-mortems – then
the non-GM fed pigs have 38 affected individuals (52% of the animals
studied), while the GM-fed pigs have 41 affected individuals (56% of the
total). Statistical significance? My ass. This is propaganda dressed up
as science, which is why it didn’t make a proper peer-reviewed journal.
(Update: Andrew Kniss makes this point better, using an appropriate
statistical technique, here.)
My judgement is that, as with Seralini, this study subjects animals to
inhumanely poor conditions resulting in health impacts which can then be
data-mined to present ‘evidence’ against GMO feeds. Most damning of all,
close to 60% of both sets of pigs were suffering from pneumonia at the
time of slaughter – another classic indicator of bad husbandry. Had they
not been slaughtered, all these pigs might well have died quickly
anyway. No conclusions can be drawn from this study, except for one –
that there should be tighter controls on experiments performed on
animals by anti-biotech campaigners, for the sake of animal welfare.
Thanks are due to the numerous Twitter correspondents who provided
insight and links which have been useful in this post. You know who you
are.
Update: I received the following expert commentaries courtesy of the UK
Science Media Centre:
Prof David Spiegelhalter, Winton Professor of the Public Understanding
of Risk at the University of Cambridge, said:
“The study’s conclusions don’t really stand up to statistical scrutiny.
The authors focus on ‘severe’ stomach inflammation but all the other
inflammation categories actually favour the GM-diet. So this selective
focus is scientifically inappropriate.
“When analysed using appropriate methods, the stomach inflammation data
does not show a statistically statistical association with diet. There
are also 19 other reported statistical tests, which means we would
expect one significant association just by chance: and so the apparent
difference in uterus weight is likely to be a false positive.”
Prof Patrick Wolfe, Professor of Statistics at University College
London, said:
“I am not an expert on animal health, husbandry, toxicology etc, and
therefore I cannot comment on these aspects of the study. As a
statistical methodologist I can however comment on the data analysis
undertaken and presented in the article.
“The biggest issue is that the study was not conducted to test any
specific hypothesis. This means that the same sample (in this case
nearly 150 pigs) is, in effect, being continually tested over and over
for different findings.
“The statistical tests employed assume that a single test is done to
test a single, pre-stated hypothesis; otherwise the significance levels
stemming from the tests are just plain wrong, and can be vastly over-
interpreted.
“Thus there is a higher-than-reported likelihood that the results are
due purely to chance. The number of pigs being in the low hundreds
(instead of, say, the thousands, as is often the case in large medical
studies) can make this effect even more prominent.
“Bottom line: a better-designed study would have hypothesized a
particular effect (such as changes in stomach size), and then applied a
statistical test solely to check this hypothesis. Perhaps another
independent team of researchers will go down this path. Until then, this
study definitely does not show that GM-fed pigs are at any greater risks
than non-GM fed pigs.”
http://www.marklynas.org/2013/06/gmo-pigs-study-more-junk-science/
First the ahem, study.
http://www.organic-systems.org/journal/81/8106.pdf
Anyway, To tear it apart Mark Lynas does a good job, saves me a bunch of
typing but came to the same conclusion:
GMO pigs study – more junk science
When I saw on Twitter that a ‘major new peer-reviewed study’ was about
to reveal serious health impacts from GMO corn and soya, I was intrigued
to say the least. Would this be Seralini 2.0, a propaganda effort by
anti-biotech campaigners masquerading as proper science, or something
truly new and ground-breaking?
Extraordinary claims require extraordinary evidence – and it would take
a lot of extraordinary evidence to confound the hundreds of studies
showing that GMO foods are just as safe as conventional, as summarised
in this recent AAAS statement:
“The science is quite clear: crop improvement by the modern molecular
techniques of biotechnology is safe. The World Health Organization, the
American Medical Association, the U.S. National Academy of Sciences, the
British Royal Society, and every other respected organization that has
examined the evidence has come to the same conclusion: consuming foods
containing ingredients derived from GM crops is no riskier than
consuming the same foods containing ingredients from crop plants
modified by conventional plant improvement techniques.”
So when I found the paper, again via Twitter, I determined to read it as
I would a climate ‘denier’ paper which aimed to overturn the scientific
consensus in that area – with an open mind, but a sceptical one. I could
see that it was already generating news, and the anti-GMO crowd on
Twitter were also getting excited about some new grist to their
ideological mill. Here’s what Reuters wrote:
“Pigs fed a diet of only genetically modified grain showed markedly
higher stomach inflammation than pigs who dined on conventional feed,
according to a new study by a team of Australian scientists and U.S.
researchers.”
Really? Time to have a look at the study. It is by a Judy Carman and
colleagues, entitled ‘A long-term toxicology study on pigs fed a
combined genetically modified (GM) soy and GM maize diet’ and published
in a minor Australian journal I have never heard of called ‘Journal of
Organic Systems’. (This journal does not appear in PubMed, suggesting it
is not taken very seriously in the scientific community. It only
publishes about twice a year, mostly with research touting the benefits
of organic agriculture.)
I skimmed the paper first, and the conclusions seemed doubtful enough
(see below) to try to find out who was beind it. So I looked at the
sponsors of this journal. They include the Organic Federation of
Australia, which seemed odd for a journal presumably aiming to be
independent. Imagine the hullaballoo if Nature Biotechnology was
sponsored by Monsanto!
I also wondered who Judy Carman and her colleagues were. She turns out
to be – as I feared – a long-time anti-biotech campaigner, with a
website called ‘GMOJudyCarman‘, which says it is supported by
GMOSeralini.org. She is a founding member, according to this website, of
the scientific advisory council of the Sustainable Food Trust. The
Sustainable Food Trust was the UK outfit set up by former organic
lobbyist Patrick Holden which stage-managed the media release of the
infamous Seralini GMO rats study to the Daily Mail and other credulous
outlets.
What about the co-authors? One is a Howard Vlieger, who seems to have
made some wild allegations about GMOs in the past if this source is to
be believed. Vlieger is president and co-founder of Verity Farms, a US
‘natural foods’ outfit which markets non-GMO grain. Despite this, the
paper declares that the authors have no conflicts of interest, although
it seems to me that he would have a very clear commercial interest in
scaring people about GMOs in order to drum up business of his GMO-free
offerings.
What about funding? The paper states that funding came from Verity
Farms, the natural product outfit mentioned above. Carman and her
colleagues are also funded by and associated with the Institute of
Health and Environmental Research, an Australian not-for-profit which
seems to be entirely dedicated to anti-GMO activism. Recent activities
have included opposing Bt brinjal in India and CSIRO’s GMO wheat in
Australia. Funding sources are not disclosed, although donations are
solicited. The paper’s acknowledgements are a veritable who’s who of
anti-biotech activism, includin Jeffrey Smith, John Fagan and Arpad
Pusztai.
So, that’s the context. Now let’s look at what raised my suspicions
about the actual study. Well, Carman and colleagues claim significant
differences in a long-term study of pigs fed GMO and non-GMO diets. But
if you look at the data they present (and the data presentation is at
least a step better than Seralini) there are obvious problems. Clearly
all the animals were in very poor health – weaner mortality is reported
as 13% and 14% in GM-fed and non-GM fed groups, which they claim is
“within expected rates for US commercial piggeries”, a vague statement
intended to justify what seem to have been inadequate husbandry
standards.
This picture is even more stark in the data presented in Table 3. 15% of
non-GM fed pigs had heart abnormalities, while only 6% of GM-fed pigs
did so. Similarly, twice as many non-GM pigs as GM ones had liver
problems. Why no headlines here? “Pigs fed non-GMO feed 100% more likely
to develop heart and liver problems, study finds” – I can just see it in
the Daily Mail. But of course negative results were not what Carman et
al were looking for.
So we fast-forward to the stomach inflammations. This is where Carman et
al got their headline. As Reuters reported:
“But those pigs that ate the GM diet had a higher rate of severe stomach
inflammation – 32 percent of GM-fed pigs compared to 12 percent of non-
GM-fed pigs. The inflammation was worse in GM-fed males compared to non-
GM fed males by a factor of 4.0, and GM-fed females compared to non-GM-
fed females by a factor of 2.2.”
This is statistical fishing of the most egregious sort, and I would put
money on the Reuters summary above being lifted near-verbatim from a
press release written by Carman et al. Table 3 actually shows that many
more pigs fed non-GMO feed had stomach inflammations than those with GMO
feed. So 31 non-GM pigs had ‘mild’ inflammation, while only 23 GM pigs
had it. For ‘moderate’ inflammation, a GMO diet again seemed to be
beneficial: 29 non-GM pigs had moderate inflammation of the stomach,
while 18 had it. So that’s 40% vs 25%. Do Carman et al perform a test
for statistical significance to see if GMO feed has a protective effect
on pigs stomachs? Of course not – that’s not the result they are after.
These findings are ignored.
Instead, it is the next line of data that they play up: for ‘severe’
inflammation 9 non-GM pigs were determined to have it, while 23 GM-fed
pigs had it. Shock, horror. You can immediately see how the data is all
over the place from the previous results, which also rule out any causal
mechanism with GMO feed – if GMO feed is causing the severe
inflammation, why is the non-GMO feed causing far more mild to moderate
inflammation? It’s clearly just chance, and all the pigs are not doing
well and suffering stomach problems: about 60% of both sets had stomach
erosion.
Yet the paper slyly presents photographs of inflamed pigs stomachs, with
non-inflamed and mildly inflamed from non-GM fed pigs, and moderate and
severe inflammation presented from GM-fed pigs. Yet 38 of the non-GM
pigs, more than half of the total of 73, were suffering moderate or
severe inflammation – why not present photos of their stomachs? This is
rather reminiscent of how Seralini presented shocking pictures of GM-fed
rats with massive cancerous tumours, but did not present pictures of the
control rats (non-GM fed) which also developed cancers.
Indeed, if you add together the ‘moderate’ and ‘extreme’ categories –
which from the photos are not easy to tell apart, involving a value
judgement on the part of the vets employed to do the post-mortems – then
the non-GM fed pigs have 38 affected individuals (52% of the animals
studied), while the GM-fed pigs have 41 affected individuals (56% of the
total). Statistical significance? My ass. This is propaganda dressed up
as science, which is why it didn’t make a proper peer-reviewed journal.
(Update: Andrew Kniss makes this point better, using an appropriate
statistical technique, here.)
My judgement is that, as with Seralini, this study subjects animals to
inhumanely poor conditions resulting in health impacts which can then be
data-mined to present ‘evidence’ against GMO feeds. Most damning of all,
close to 60% of both sets of pigs were suffering from pneumonia at the
time of slaughter – another classic indicator of bad husbandry. Had they
not been slaughtered, all these pigs might well have died quickly
anyway. No conclusions can be drawn from this study, except for one –
that there should be tighter controls on experiments performed on
animals by anti-biotech campaigners, for the sake of animal welfare.
Thanks are due to the numerous Twitter correspondents who provided
insight and links which have been useful in this post. You know who you
are.
Update: I received the following expert commentaries courtesy of the UK
Science Media Centre:
Prof David Spiegelhalter, Winton Professor of the Public Understanding
of Risk at the University of Cambridge, said:
“The study’s conclusions don’t really stand up to statistical scrutiny.
The authors focus on ‘severe’ stomach inflammation but all the other
inflammation categories actually favour the GM-diet. So this selective
focus is scientifically inappropriate.
“When analysed using appropriate methods, the stomach inflammation data
does not show a statistically statistical association with diet. There
are also 19 other reported statistical tests, which means we would
expect one significant association just by chance: and so the apparent
difference in uterus weight is likely to be a false positive.”
Prof Patrick Wolfe, Professor of Statistics at University College
London, said:
“I am not an expert on animal health, husbandry, toxicology etc, and
therefore I cannot comment on these aspects of the study. As a
statistical methodologist I can however comment on the data analysis
undertaken and presented in the article.
“The biggest issue is that the study was not conducted to test any
specific hypothesis. This means that the same sample (in this case
nearly 150 pigs) is, in effect, being continually tested over and over
for different findings.
“The statistical tests employed assume that a single test is done to
test a single, pre-stated hypothesis; otherwise the significance levels
stemming from the tests are just plain wrong, and can be vastly over-
interpreted.
“Thus there is a higher-than-reported likelihood that the results are
due purely to chance. The number of pigs being in the low hundreds
(instead of, say, the thousands, as is often the case in large medical
studies) can make this effect even more prominent.
“Bottom line: a better-designed study would have hypothesized a
particular effect (such as changes in stomach size), and then applied a
statistical test solely to check this hypothesis. Perhaps another
independent team of researchers will go down this path. Until then, this
study definitely does not show that GM-fed pigs are at any greater risks
than non-GM fed pigs.”
http://www.marklynas.org/2013/06/gmo-pigs-study-more-junk-science/
Comment