California Water and Environmental Modeling Forum
Promoting Excellence and
Consensus in Water and Environmental Modeling
www.cwemf.org · (916) 607-0435
California Water and Environmental Modeling Forum
Promoting Excellence and
Consensus in Water and Environmental Modeling
www.cwemf.org · (916) 607-0435
Technical Workshop
on
Identifying and Managing Uncertainty
in Ground and Surface Water
Models
A one-day course for
managers, regulators and those who rely on the outcomes of models.
Monday, April 28, 2003
9:00 a.m. to 4:00 p.m.
1001 “I” Street, Sacramento,
CA
Internet audio broadcast at:
www.cwemf.org
No-charge
for CWEMF members; $50 for non-members; Advanced registration not required.
The
time has passed when modeling of environmental processes can take place without
an analysis of the uncertainty associated with model predictions. Furthermore,
nowhere is the requirement for model predictive uncertainty analysis greater
than where models are used in situations of disputation. No longer can a model
that pretends to be a crystal ball, and purports to provide a unique and
unequivocal prediction of environmental system behavior, stand up in court. Nor
can such a model have the integrity required to form the basis for negotiation
between different stakeholder groups.
Those
who build models, and those who depend on them, should understand the strengths
and limitations of computer simulation of environmental processes. The use of
modern environmental models in conjunction with sophisticated parameter
estimation and predictive analysis software allows environmental data to be
processed, and hypotheses to be tested, like never before. However, part of
modern modeling practice must be the recognition and accommodation of
situations where two or more differing (sometimes conflicting) hypotheses
cannot be rejected based on the currently available dataset - for example, the
location of contaminant sources, or the outcome of a certain watershed
management strategy. Recognition and definition of the limits of a model’s
ability to make definitive predictions will allow dispute resolution to take
place constructively on the basis of quantified predictive uncertainty, rather
than destructively and acrimoniously on the basis of conflicting and misleading
models, each with flimsy facades of certainty, brought to the courtroom or
negotiating table by different stakeholder groups.
It
is through the process of model parameterization
that an off-the-shelf software package (such as MODFLOW or HSPF) becomes a
purpose-built environmental simulator for a particular aquifer, basin or
watershed. Parameter assignment is often achieved through model calibration –
the process of “tuning” a model to a particular environmental system. However
the constraints on parameters imposed by the calibration process are often only
weak (even when data appears to be plentiful); hence the extent of parameter
nonuniqueness is often frighteningly high. Even moderate uncertainties in
parameter values can, under some circumstances, be translated into large
uncertainties in model predictions. Unless this uncertainty is quantified, the
modeling process is incomplete at best and misleading at worst.
This unique course attempts to explain how uncertainty is introduced to a model through the parameterization and calibration process, and the steps that can be undertaken to recognize this uncertainty, and incorporate it into model predictions The course is aimed at the entire spectrum of those who work with models – those who build them, those who manage (or are managed) on the basis of model results, those who takes models to court. No prior modeling experience is required; however even seasoned modelers will find this course interesting and informative as this often-neglected aspect of modeling is discussed in detail with many practical examples.
Anyone
with an interest in groundwater or surface water modeling will benefit from
this course. The principals, rather than the details, will be presented; so no
modeling experience is required. Regulators, managers, lawyers, and modelers
themselves will all learn from this part of the course.
John
Doherty is the author of PEST (www.sspa.com/PEST/),
the industry standard in environmental model calibration software. John has been in the water industry for over
25 years, first as a geophysicist and then as a modeler. He has worked in the public,
private and education sectors. He now directs his own company, Watermark
Numerical Computing, which undertakes software development and advanced
modeling for mining, environmental, agricultural, water supply and remediation
applications. He also works as a senior research scientist for the University
of Queensland where he and his students undertake research into methods of
environmental model predictive uncertainty analysis. John has had over eight years experience in presenting short
courses all over the world. Course material is presented clearly and
descriptively with many practical examples and illustrations. He attempts to
create a learning environment that is both educational and enjoyable. Note: Two case studies will be presented by Jim Rumbaugh
(Environmental Simulations International) and Matt Tonkin (Ph.D. student of
John Doherty).
·
Why
modeling is not like gazing into a crystal ball.
·
The
“reasonable assumption” problem - why many model are built on foundations of
sand.
·
Some
embarrassing model failures.
·
The
allure of complexity.
·
Why
complexity often hinders, rather than assists, use of the modeler’s intuition
in the modeling process (and why that intuition is so important).
·
Rigid
models vs. flexible models.
·
Mean
and standard deviation
·
Normal
and other distributions
·
Independent
random variables
·
Correlated
random variables
·
Covariance
matrix and functions of the covariance matrix
·
Parameter
insensitivity
·
What
is “calibration”
·
Why
a model needs to be calibrated
·
Very
short theory of parameter estimation
·
Characterization
of model parameter uncertainty
·
Why
it is easier to estimate combinations of parameters than individual parameters
·
Examples
of parameter nonuniqueness in ground and surface water modeling
·
Complexity
and the need for extra parameters
·
What
regularization means
·
The
desirability of parameter uniqueness
·
Manual
regularization
·
Software-based
regularization
·
Pilot
points as a method of spatial parameterization
·
Lumped
parameters as a regularization device
·
Examples
of regularization in geophysical, groundwater and surface water modeling
·
The
cost of regularization – loss of system detail and (possibly large) bias in
model predictions
·
The
fundamental choice in environmental modeling – uniqueness or
over-parameterization
·
Why
many modeling exercises are doomed from the moment of inception.
·
Sensitivity
analysis
·
Predictive
analysis while recognizing calibration constraints
·
Linear
uncertainty propagation – UCODE and MODFLOW2000
·
Nonlinear
uncertainty propagation
·
PEST’s
predictive analyzer
·
Examples
of nonlinear predictive analysis from ground water modeling
·
Examples
of nonlinear predictive analysis from surface water quantity and quality modeling
·
Predictive
analysis as a calibration constraint
·
Modeling
as a means of hypothesis testing
Accommodation of unknown detail – the repercussions
of detail that might exist
·
Why
important detail may exist which is beyond the ability of the calibration
process to infer
·
The
repercussions of this detail for model predictive uncertainty analysis
·
The
importance of intuitive knowledge in limiting the range of predictive
uncertainty
·
Monte
Carlo analysis
·
Imposing
calibration constraints in Monte Carlo based parameterization.
·
Markov
Chain Monte Carlo analysis
·
Nonlinear
predictive analysis in over-parameterized systems
·
Examples
of predictive analysis in over-parameterized systems
·
Costs
and benefits of model complexity
·
Basic
theory of model complexity
·
Complexity
and uncertainty
·
Examples
– ground and surface water modeling
·
Defining
what is achievable in modeling through predictive uncertainty analysis
·
Optimization
of data acquisition
A
number of examples, based on experience gained by the course instructors, will
be presented to illustrate the need for the accommodation of uncertainty in the
presentation of model results, and how this was achieved in real-world modeling
contexts.
·
Parameterization, Uncertainty and
“Real World” Modeling (UncertaintyMain.ppt; 52 KB)
Building: From
I-5, take the “J’ Street offramp heading East and turn left on 10th
Street. Proceed North for one
block. Parking: Lots located at 11th and I ($1 each ½
hour, $12 daily max), 13th and I ($unknown), and 8th and
I ($1.50 each hour); 10-hour meters (uses up to 14 quarters) are located on 10th
St. between H & I, on G St. between 11th & 12th,
and on F St. between 7th & 10th. Room:
Go up the stairs to the 2nd floor and do a U-turn turning right; the
Sierra Hearing Room is straight ahead.
The Cal EPA Building “Sierra” Hearing Room is accessible
to persons with disabilities. Individuals
who require special accommodations are requested to call Adrian Perez at
916-341-5880 at least five working days before the workshop date. TTY users may contact the California Relay
Service at 1-800-735-2929 or voice line at 1-800-735-2922.
A
three-day course entitled “Model Calibration and
Predictive Uncertainty Analysis using PEST,” which includes hands-on
applications, is being offered April 29 to May 1, 2003 in San Francisco,
CA. For more information, please visit the Groundwater Resources
Association of California website at: www.grac.org/pest/.