WASHINGTON (Saving Seafood) — June 10, 2013 — The Conservation Law Foundation (CLF) recently criticized as a “flat-out falsehood” Saving Seafood’s description of the Lenfest Forage Fish Task Force’s report as lacking peer review. Subsequently, the Pew Charitable Trusts made the same argument in a letter to the editor of Seafood News. Saving Seafood would like to clarify our statement referencing the “unproven assumptions, as well as a lack of peer-review of the Lenfest Forage Fish Task Force.”
When we labeled the Task Force’s report, “Little Fish, Big Impact,” as absent peer review, we were referring to the report’s recommendations, specifically that “forage fish” harvests should be severely restricted. Neither those recommendations, nor the analysis supporting them, have been published in a peer-reviewed scientific journal.
Both CLF and Pew stated that the report’s acknowledgements “clearly” indicate that it was peer-reviewed by three independent reviewers. But the report only discloses that “three anonymous reviewers lent their time and expertise to shaping the final form” of the report. “Shaping the final form” and “peer review” are not synonymous.
When contacted for this article, a Task Force member stated that some of those involved in the Task Force considered this anonymous, in-house review to be the equivalent of a formal peer review process. We were also told that the report's anonymous reviewers were in fact selected by Pew, which manages the Lenfest Ocean Program that convened the Task Force.
According to the Pew Charitable Trust’s website, the Lenfest Ocean Program “was established in 2004 by the Lenfest Foundation and is managed by Pew.” A Pew press release earlier this year referred to the “forage fish task force, convened by the Pew-managed Lenfest ocean program…”. The Policy Advisor of the Task Force, Chris Mann, is a "senior officer" with the Pew Environment Group, where he works on “forage” fisheries, among other responsibilities. Christine Santora, the Task Force Project Director, previously worked as a Senior Research Associate with the Pew Institute for Ocean Science.
When a paper is submitted to a journal for conventional peer review, the journal’s editorial staff selects independent anonymous reviewers, who can accept, reject, or offer suggestions regarding the paper. The American Fisheries Society, in a 2006 document “Defining and Implementing Best Available Science for Fisheries and Environmental Science, Policy, and Management,” lists five goals for a peer review process: “the peer review covers (1) the validity of the methods used, (2) whether the methods and study design adequately address the objectives, (3) whether the results that are reported are adequate for interpretation, (4) whether the results support the conclusions, and (5) whether the findings represent a significant advance in scientific knowledge.”
In the case of the report of the Lenfest Task Force, it is not clear how extensive the process of “shaping the final form” was, whether this review was consistent with the best practices of an independent review, or whether the reviewers had the right to reject the report’s findings and recommendations. But even if the review met industry-standard practices, the reviewers themselves were selected by the same organization that convened the Task Force, which is clearly not reflective of an independent selection process.
While we do not question the intentions of those who worked on “shaping the final form” of the report, we stand by our assessment that the report’s recommendations and non-economic analyses, which have not been published in any scientific journal, were not submitted for an independent peer review. Given the circumstances surrounding the management of the Lenfest Task Force, and the role of the Task Force’s managing organization in selecting reviewers, we find CLF’s characterization of our statement as a “flat-out falsehood” to be a flat-out falsehood. The circumstances surrounding the Task Force report review process are not black and white, but grey at best.
However, we want to be clear. One aspect of the Task Force’s work has undergone peer review. Specifically, the Task Force members drafted a paper, “The global contribution of forage fish to marine fisheries and ecosystems,” published in the journal Fish and Fisheries. This paper dealt with the relative economic value of “forage fish” as direct catch vis-à-vis their “supportive value” for predator species. But it did not include the Task Force’s recommendations for species management.
Saving Seafood’s primary point remains focused on the Task Force report, which has not undergone any outside review by objective third parties. Because its reviewers were selected by the report’s funder, the Pew Environment Group, the report has not faced the standards for objective third-party peer review generally provided by independent scientific journals. If the Task Force's recommendations on "forage fish" are to become the national standard, extensive review by regulators and third party experts should be required.
We stand by our position.