Such models are often a crucial part of the
information analysis supporting regulatory decisions, an agency source says,
because they help agency experts use existing data to "make decisions about
future events." In some cases they have come under attack from industry groups
that say models can be misleading or meaningless, leading to inaccurate
projections of likely environmental outcomes and therefore unrealistic program
targets and regulations.
The draft report could help EPA overcome such
challenges by providing clear recommendations on how to use valid models and
ensure they result in justifiable regulatory decisions, the source says. The
report will help provide the agency with "guidelines and a vision of the
selection and use of models" in future regulatory decisionmaking, according to
the panel's Web site. The agency source says the draft NAS report will be sent
to EPA at the end of this month.
EPA is also awaiting a report from NAS on how the
agency should revamp its toxicological testing system for pesticides and
industrial chemicals, which could replace the testing protocols EPA has used
since the 1980s, a second agency source says.
NAS convened the scientific models panel in
November 2003 at EPA's request to provide advice on ensuring models are of
high quality and lead to justifiable decisions. A main impetus for that request was the 2001 passage of
the Data Quality Act, the first EPA source says, which allowed outside parties
to challenge the accuracy of information EPA disseminates for public
use. Among the first challenges were a
petition from the Competitive Enterprise Institute questioning the validity of
EPA using climate change models as the basis for its 2002 global warming
action report and a broad U.S. Chamber of Commerce challenge to EPA model
"A crucial question when we started [requesting
NAS' assistance] was how can we say our models lead to credible decisions?"
the first EPA source says, adding that despite frequent uncertainties due to
lack of data or untested hypotheses, "we have to make a decision" when
regulation or program development is called for. The NAS report is meant to
give EPA direction on how to ensure that use of models, such as contaminant
dispersion models or epidemiological models, leads to justifiable regulations
in the future, the source says.
The report will not be narrowly tailored to a
single model or type of model, the source says. The panel members' expertise
varies widely -- from law to chemical engineering to the history of scientific
processes -- which will help the report give a range of suggestions and
recommendations on how to think about information quality and how modeling
results lead to decisions, as well as more technical aspects of the issue, the
Once EPA has reviewed the draft, it will take the
panel's recommendations and combine them with suggestions already offered by
the agency's Science Advisory Board (SAB) last August to produce a new draft
of the November 2003 "Draft Guidance on the Development, Evaluation, and
Application of Regulatory Environmental Models," the source says. Among other
recommendations, SAB said the guidance should more fully address "the need to
develop and apply models within the context of a specific problem," should
observe "[c]aution in the way that information on modeling uncertainty is
evaluated and communicated," and should "more fully discuss uncertainty and
sensitivity analysis methods." Relevant documents are available on
Further, SAB noted in its review of the draft
modeling guidance that the vision of the Regulatory Environmental Modeling
(REM) program, which drafted the guidance, is "not matched by a commensurate,
and steady, allocation of resources on the part of the Agency. It is therefore
recommended that the Agency provide a meaningful commitment of resources to
the REM initiative."
An EPA spokeswoman says in a statement that the
agency "has taken the . . . guidelines very seriously and has put in the
resources necessary to meet the spirit of the guidelines." The EPA Office of
Research & Development "is funding the coordination expenses (i.e.
seminars, workshops, and the centralized knowledge base)," the statement says
without providing financial or other details.
The new guidance, when it is drafted, will be
significant because "it articulates what we think of model evaluation vetting
at EPA," the agency source says, predicting that it "will permeate throughout
the agency." What is most important about the guidance is to have "an idea of
what to do in evaluating models" that everyone can refer to, the source
The guidance will seek to describe how the
principles of sound science should be addressed in vetting the use of models;
to encourage peer review of a proposed model along with a data set to test its
validity; and to examine how models meet the objectives and standards of
quality assurance staff at EPA. Among the most important issues in the science
of modeling is deciding which model could not only give plausible results but
could best describe the real-world conditions researchers are attempting to
understand, the source says. "You can come up with four plausible models," the
source says, but it is important to decide "which one should you
The source could not say when a new draft of the
guidance would be available.
NAS is also preparing to release a separate report on what a future system for
conducting toxicological testing should look like. The report -- due to be
released in the middle of June, according to an NAS source -- will seek to
answer several questions EPA charged the committee with addressing in April
2004, including the benefits potentially arising from new testing
technologies, the challenges to achieving them, and approaches and incentives
that might be used to address the challenges.
The report will hopefully lay out a new testing
system to replace the paradigm laid out in the 1980s for examining the
toxicity of pesticides and the risks of industrial chemicals regulated under
the Toxic Substances Control Act, the second agency source says. That system
was created before many recent advances in data collection and analysis had
been made, the source says, adding that the upcoming report will seek to lay
out NAS' "vision" for an up-to-date way to take advantage of that
Such questions are "very important to the
pesticide program," for instance, because of the number of substances to be
tested and regulated and because of the cost of doing expensive animal tests,
the source says. "We shouldn't be asking for animal studies we don't need. The
1980s paradigm was good for that time, but we can design something
Among the newer tools likely to be discussed in
the report are so-called quantitative structure activity relationship models,
which can relate a chemical's biological activity, including its
carcinogenicity, to its structural components, according to EPA's Web site.
Such techniques are being increasingly used to identify chemicals and other
substances most likely to be harmful to human health to narrow the field of
potential subjects for more extensive tests, the source says.
An EPA spokeswoman did not respond to a request
for comment on the report.