Pyrkilo

From Opasnet
Jump to: navigation, search

<section begin=glossary />

Pyrkilo is a name by which the collaborative assessment method-tool compound, currently known as open assessment method and Opasnet workspace, used to be called in early phases of its development. See open assessment and Opasnet.

<section end=glossary />

This page contains a compilation of descriptive texts on Pyrkilo. It considers both the methodology of (risk) assessment and the tool to support the use of the method. These texts primarily have a historical purpose, up-to-date descriptions and discussions on the topic are found under the terms open assessment and Opasnet.


The current definition of pyrkilo is the following:

Pyrkilo theory is a theory that attempts to answer the following research question: How can scientific information and value judgements be organised for improving societal decision-making in a situation where open participation is allowed?

Pyrkilo method is a method that applies the contents of the pyrkilo theory in practical risk assessments.

Open risk assessment is a risk assessment (covering both the process of making it, and the final product) that has been carried out utilising the pyrkilo method.

Some of the terms used are (tentatively) defined below.

  • Scientific information = information that can be obtained by using the scientific method [1]
  • Value judgement = preference or value a person or a group assigns to a particular condition or state of the world
  • Societal decision-making = process of reaching conclusion about actions that can be taken by the institutes of the society, or the members of the society in general, in aim to increase societal values
  • Societal values = value judgements that are so widely accepted in the society that they can be thought of representing a large part (not necessarily majority) of the members of the society
  • Improving decision-making = practices that, on average, lead to decision-making that, on average, leads to outcomes with higher societal values than an alternative practice
  • Open participation = a process where a non-organised, non-fixed, non-restricted group of individuals can get information from and contribute to the outcome of the process (in this context, a risk assessment report).

Pyrkilo is currently most actively used in the field of risk assessment. Pyrkilo as a method differs from the other risk assessment methods in three essential aspects:

  1. The risk assessment product IS a causal diagram where all elements are described as variables using a specified structure. This sets structural limits to the contributions. In traditional risk assessments, the assessment product typically is a report using a structure of a book, not that of a causal diagram. The causal diagrams are used for illustration only, or within models that calculate some results to the assessment report.
  2. Formal argumentation (according to the pragma-dialectical argumentation theory) is used as the primary means to describe and resolve any disputes about scientific or valuation issues within the assessment. In traditional risk assessments, there is guidance to describe major disputes, but there are no structural rules for this. In addition, many disputes are (implicitly) resolved using conventions without challenging the foundations of the convention.
  3. The principles of mass collaboration are used in the risk assessment work. In practice, anyone is allowed to contribute (i.e. bring in information and value judgements that are within the scope of the assessment), unless there are explicitly specified reasons to restrict participation in the particular case. The traditional risk assessments are expert-driven exercises with possibly some public hearings about the scoping or results, but little involvement of the stakeholders to the actual making of the assessment.

References

  1. Popper, Karl R. 1935. Logik der Forschung. Julius Springer Verlag.

Intarese method is a method for performing environmental health risk assessments in an efficient way to produce high-quality quantitative assessments for improving societal decision-making. The aim is to improve the making of risk assessments by offering guidance, tools, and examples. Another aim is to improve the availability and usability of the assessments by offering assessments and other contents for anyone to read, learn, and apply. The help pages that describe parts of the method in detail belong to the Category:Intarese method, and links to them can also be found from there. D↷

The ultimate purpose is to improve societal decision-making.

← The Intarese method helps in this by providing relevant information in the form of quantitative assessments about risk situations.
← In general, the information provided is about predictions on the impacts of possible decisions on some outcomes that have a societal value. However, the scope of the information can be narrowed in a particular case for example in the following ways:
  • The only decision considered is business-as-usual. This results in estimates of the burden of the current situation.
  • Outcomes considered are not explicitly valuated, but implicitly it is assumed that they relate to something that has intrinsic value. For example, pollutant concentrations can be estimated, although it is the related health impact that has the intrinsic value, and the concentration is only a proxy of this.

The Intarese method is closely related to the pyrkilo method that is being developed in KTL, Finland. Template:Pyrkilo definition

Basic ideas of the method

The structure of a risk assessment is completely rethought compared with the established NRC (1983) concept described in the "Red Book":

  • A risk assessment consists of variables. Each of them describe a particular piece of reality. Importantly, variables are independent of the risk assessment they belong to, and each variable can be used is several risk assessments.
  • A variable is also independent of all variables except those that are mentioned in its definition attribute. (These are called the upstream variables.) Thus, the result of a single variable can be calculated if its definition is described in a precise enough way and the results of the upstream variables are available.
  • Thus, a risk assessment does not consist of a single risk assessment model to calculate the outcome results. Instead, it consists of several integrated models, one for each variable needed in the assessment. This approach has several important implications.
    1. The risk assessment work can be effectively decentralized after the essential variables have been identified.
    2. The actual computing tasks can also be decentralized. They can be performed using any relevant software, as long as there is a standardized interface and information structure between the variable description system and the result computing system. This interface is called Variable transfer protocol (VTP). This also makes it possible that the model and computing effort of a variable can be very large, and still it does not cause severe load on the toolbox that maintains the risk assessment work (see below).
    3. The variable results, if collected in a centralized way, is a valuable database for making further risk assessments and utilising the information in decision-making.
    4. The structure gives freedom to further development, as the different parts of the whole system can be operationalized using the software and programs best suited for that purpose, as long as the interfaces between them follow the VTP.
  • To be able to utilise the risk assessment method, a toolbox must be developed. The toolbox is a collection of software and functionalities that provide the guidance, information structure, and data for the participants of a risk assessment. The toolbox functionalities are described in more detail below.


The main parts of the toolbox

Intarese toolbox is a software for applying the Intarese method. Details of is are shown on the graph below. Click on a detail to get more specific information.

  1. Collaborative workspace
  2. Separate calculation tools
  3. External models
  4. Result database
  5. Analysis tool

Template:Toolbox and its functionalities

The phases of a risk assessment

The risk assessment work has six phases. The work during a particular phase is always built on the results of the previous phases. The phases are not clear-cut periods in time, because the work is iterative in nature, and the previous phases must frequently be revisited when the development of the product and feedback brings in new understanding. Thus, the next phase is usually started when the previous phase is still a draft at least in some aspects.

The phases of risk assessment are called:

  1. Scoping of the assessment. In this phase the purpose of the assessment, the question(s) asked in the assessment, intended use of the output, the temporal and spatial boundaries of the scope of the assessment, and the participatory width of the process are defined and described.
  2. Applying previous information about the issues being assessed. In this phase the existing information that is available e.g. in the encyclopedia, as existing variables from previous assessments, or as Classes (general properties shared by a group of variables) are sought out and applied.
  3. Drawing a causal diagram. In this phase the decisions, outcomes, and variables of importance related to the assessment are described in the form of a sketch of a causal network diagram.
  4. Designing variables. In this phase the variables included in the assessment (the causal network diagram) are described more precisely, including defining causal relations between them. This phase may also include definition of quality criteria and plans for collecting the necessary data or models to estimate the results of the variables. It should be noted that previously this phase was considered to apply to the assessment as a whole. Now we have realised that in practice the design work happens separately for each variable (although usually approximately at the same time). The same applies to the next phase, execution.
  5. Executing variables and analyses. This phase is actually about collecting the data needed, executing the models described in the Definition attributes of different variables, and storing the results in the result database. Assessment-specific analyses such as optimisation, decision analysis, value-of-information analyses and so on are carried out.
  6. Reporting In this phase, the results of indicators and assessment-specific analyses are communicated to the users of the outputs. The results are discussed and conclusions are made about them, given the scope of the risk assessment. The communication includes necessary background information.

The phases 1-3 are collectively called the issue framing phase.

Structural elements of the method


Crucial processes when using the method


See also

There are eight challenges that are especially emphasized

  1. Help:Purpose of a risk assessment
  2. Help:Causality
  3. Help:Variable
  4. Help:Mass collaboration
  5. Help:Value judgement
  6. Help:Dispute
  7. Help:Applying general information
  8. Help:Copyright issues



  • PyrkiloD↷ theory is a theory that attempts to answer the following research question:
    How can scientific information and value judgements be organised for improving societal decision-making in a situation where open participation is allowed?
  • Pyrkilo method is a method that applies the contents of the pyrkilo theory in practical risk assessments.
  • Open risk assessment is a risk assessment (covering both the process of making it, and the final product) that has been carried out utilising the pyrkilo method.


The fundamentals of open risk assessment and the pyrkilo method:
  1. let everyone speak,
  2. try to understand their reasoning in respect to the real world,
  3. act based on the synthesis.

Pyrkilo as a method differs from other risk assessment methods in three essential aspects:

  1. The risk assessment product IS a causal diagram where all elements are described as variables using a specified structure. This sets structural limits to the contributions.
  2. Formal argumentation (according to the pragma-dialectical argumentation theory) is used as the primary means to describe and resolve any disputes about scientific or valuation issues within the assessment.
  3. The principles of mass collaboration are used in the risk assessment work. In practice, anyone is allowed to contribute (i.e. bring in information and value judgements that are within the scope of the assessment), unless there are explicitly specified reasons to restrict participation in the particular case.
  4. Value judgements are included in the assessment as essential and explicit parts. They are described as variables, and they have the same structure and properties as scientific information. However, they change in the way how information can be obtained and included: when a person expresses his or her own value judgement, the content is valid as such.

Pyrkilo method approaches risk assessment as interpretation and combination of scientific information and value judgments for use of the society. Risk assessment is thus an activity that takes place in the interface between science and society. Risk assessment consists of a process of making it, and the resulting product. The product (risk assessment report) consists basically of the question(s) to be asked, the causal diagram about the risk situation, and the results and conclusions based on the causal diagram.

The purpose of a risk assessment is to improve societal decision-making in a particular risk situation.

← The assessment should provide relevant information about the risk situation in a quantitative form.
← In general, the information provided is about predictions on the impacts of possible decisions on some outcomes that have a societal value.


It is possible to define the general properties of good risk assessments according to the following three different categories:

  • Quality of assessment product content
  • Applicability of the assessment product
  • Efficiency of the assessment process

Quality of content refers to the goodness of the description of reality that is produced in the assessment. It consists of the properties called informativeness, calibration and internal relevance.

Applicability consists of the properties called external relevance, usability, availability and acceptability. Acceptability can be further defined as acceptability of premises and acceptability of the assessment process. The applicability properties become realized in the use of risk assessment product.

Efficiency can be divided into two properties, intra-assessment efficiency and inter-assessment efficiency. The previous refers to efficiency of the process within a single assessment and the latter to efficiency of the process in making a series of assessments.


The assessment product, a causal diagram, consists of variables, which are descriptions of particular pieces of reality. It should be noticed that, according to the principle of open risk assessment, the variables are NOT owned by any single risk assessment, but they can be freely used in any risk assessment where they are relevant. Because of this, there is a special need to describe variables in a way that they are self-explanatory and self-calculable (as long as the upstream variables are known in the causal chain).

Variables can be used for describing several kinds of things, e.g. physical phenomena, value judgments and decisions. All kinds of variables are defined using the following attribute structure described in the table below:


The process of carrying out a risk assessment can be considered as consisting of four simultaneously on-going sub-processes that are continuous throughout the whole risk assessment:

  1. Collection of scientific information and value judgements
  2. Synthesis and manipulation of scientific information and value judgements
  3. Communication of the outcomes of collection and synthesis
  4. Management of collection, synthesis and communication sub-processes

The assessment process progresses from defining the assessment questions to providing sufficiently complete answers in iterative phases as follows:

  1. Scoping of the assessment: Defining the purpose, question(s), intended use, boundaries, and the participatory width of the process.
  2. Applying previously created readily available information about the issues being assessed.
  3. Drawing a causal diagram including decisions, outcomes, indicators and other variables.
  4. Designing variables: Defining the attribute contents for individual variables.
  5. Executing variables and analyses: Collecting the data needed, executing the models, defining the results of variable and making assessment-specific analyses.
  6. Reporting the assessment: Communicating the results and conclusions to the users.

The phases 1-3 are collectively called the issue framing phase.


Open risk assessments are challenging things to accomplish. Even a single-user work on risk assessments is complex. When a non-restricted group of people is allowed to participate, there are specific technical problems to manage the process, not to mention the contents. Therefore, tools have been developed to facilitate open risk assessments.

Necessary parts of an open risk assessment system are:

  • Process management system
  • Content management system
  • Collaborative workpace

The process management system gives guidance for the work and offers tools to make risk assessments, collect information, and manage the contents.

The content management system contains information and data about environmental health issues and risks. The products of risk assessments (assessment reports and variables) are located here.

Collaborative workspace is a virtual working platform that allows open groups to participate in risk assessments. It is an interface for the users to access the system contents, make their contributions to the assessment and communicate between each other.


Key words: open risk assessment, pyrkilo, environmental health, stakeholder participation, argumentation, mass collaboration, societal decision-making


Template:Release The Intarese general method is a procedure that helps make better environmental health risk assessments. It consists of a coherent set of methods and guidance that covers all the phases and steps of carrying out an environmental health risk assessment. Many of these methods are commonly used and well established, some exist but are too rarely used, and some have been and are being developed in Intarese specifically for this purpose, to fill the gaps that have been identified in the risk assessment process. All these methods together provide the means for the risk assessors to carry out good quality environmental health risk assessments. The specific methods included in the Intarese general method provide help, information, suggestions, and defaults for the risk assessors in making their specific assessments. Intarese general method is the main product of the Intarese project.

Introduction

This page provides an overview and brief descriptions of all the specific methods belonging to the Intarese general method. The specific methods are grouped under different categories according to the scope of influence that the methods are intended for. This page functions also as an index providing links to more detailed descriptions and guidance of each particular method and this page can be used as the main page or entry point to all Intarese general method relevant pages. All these relevant pages are also categorized to the Intarese general method category. If a specific method is included in the list, but there is no link, it means that the method has been identified as important, but the method is still lacking a description.

The list of specific methods is not yet fully comprehensive and the groupings of the specific methods under different categories may not yet be quite perfect. However the page already provides a good overview to the Intarese general method and illustrates the different roles of different specific methods in carrying out a full integrated risk assessment.

This page also describes briefly the seven fundamental new elements that have been identified during the method development work in Intarese project. These seven elements are also included in the Intarese general method. These elements are issues that, when used together, result in completely new kinds of risk assessments and new ways of making better risk assessments. None of these seven elements are completely new as such, but currently they are not commonly used in a tight combination in carrying out risk assessments. These elements are described in brief below. More thorough descriptions are included in the descriptions of the specific methods they are related to.

For clarity, some definitions:

  • Phase = a certain stage in the risk assessment process
  • Step = a certain stage in the causal chain
  • Variable = an object described as a part of the causal chain description
  • Indicator = a variable which is of special interest

The seven plus one fundamental new elements in the Intarese general method

1. Purpose: Better societal decisions
Risk assessments should always be done for a purpose. When the purpose is identified and kept clear in mind and preferrably explified and made public, it helps to guide the process in producing a desired kind of assessment product. The ultimate general purpose is to improve societal decision making by providing good descriptions of chosen parts of reality for the use of the decision-makers. The process of describing the purpose is now considered as an essential and integral part of the method. Proper identification of the purpose of risk assessment crucially affects the assessment process and the content and essence of the final product. For further details, see General properties of good risk assessments.

2. Causality
The Intarese method is based on the full-chain approach. This implies that the assessment products produced in the assessments should be causal network descriptions that cover the relevant phenomena from emissions to exposures to health effects and their impacts in accordance with the chosen endpoints and purpose. However, it should be emphasized that the method does not only describe issues that are associated with the full chain. It describes those issues that cause effects along the full chain, and it describes how the causes and effects are related. This, of course, makes risk assessment a challenging, or even difficult, process. Strict emphasis on causality, however, should be the way to e.g. estimate the impacts of policies on emissions and consequently to health effects. For further details, see Guidance and methods for indicator selection and specification.

3. Collective structured learning
Science is inherently an area that is based on learning from previous experience, and this is not a new concept. However, collective structured learning means something more than that. It is an idea based on intentionally producing generally applicable pieces of information that can also be used as the building blocks of new assessments. (In discussions within Intarese, these have been referred to as e.g. templates, objects, or general variables, but a common term has not yet been established.) In addition, collective structured learning builds on the principle of collaborative work, which implies that the pieces of information produced in a particular assessment are set commonly available for use in other assessments. What is new in this concept is that there are specific procedures for extracting pieces of information from previous work and assessments, and making them centrally available for others. This enhances the efficiency and effectiveness of the assessments made by applying the Intarese general method. For further details, see Collective structured learning.

4. Value judgements
Risk assessment is about estimating impacts that have positive or negative value judgements attached to themselves or to the determinants that causally affect them. These values must be acknowledged in the process of making the assessments, not only in the risk management phase, otherwise there is a risk of compromising the relevance of the assessment. Combining phenomena of physical reality with the value judgements related to them requires methods to distinguish these two things from each other and bringing the value judgements to explicit scrutiny within an assessment.

5. Variable structure
In order to make coherent descriptions of reality in assessments, there must be a certain clear structure within the assessment. As we also want to produce descriptions that coherent also between assessments, there must be a universal structure for all the assessments. Variables with a certain set of attributes, and linkages between these variables are the universal structure of the assessments. For further details, and the current suggestion of the attributes, see Guidance and methods for indicator selection and specification |. The universal assessment structure is essential for coherent inclusion of causality in assessments, enabling of collective structured learning, collaborative work as well as combining value judgements with descriptions of physical reality.

6. Collaborative work
Collaborative work means more than just dividing tasks within a group into pieces that belong to someone. It is a new way of working together on a shared set of tasks for a common goal. Its best properties can be seen in situations where there is a diverse (and maybe unknown) group of potential participants whose knowledge and value judgements on the issues relevant to the particular assessment are combined in the description. With the aid of fairly simple rules, this group can work with little or no central guidance and still gather rapidly relevant information. The Intarese general method should contain these rules in its aims, so as to be able to facilitate stakeholder involvement and the collection and synthesis of information. To facilitate such synthesis, risk assessments are carried out in an assessment workspace which functions as the platform for collaborative work. The purpose of the assessment workspace is to offer an interface for accessing and applying all the specific methods included in the Intarese general method. Even more importantly, its purpose is to function as a platform for carrying out and managing the assessments in one virtual location. This feature of assessment workspace also enhances the possibility to extract and re-use information from the previous assessments that have been performed using the workspace. For further details, see Collaborative work | Managing stakeholder involvement | Pyrkilö-wiki vision | Assessment workspace (Intarese method) | Intarese toolbox.

7. Dealing with disputes
When a diverse group of contributors participate in making a risk assessment, it is obvious that disputes may arise. One of the most instructive features of risk assessment is to understand both these disputes and the reasons why a particular outcome occurs. Intarese general method must include methods that help to deal with disputes, find resolutions and document the choices made so that they can be defended afterwards. Argumentation theory offers a basis for these methods. For further details, see a manuscript dealing with this issue.

+1. Information repository in public domain
A large part of the costs of making a risk assessment arises from collecting basic information. There are large quantities of data available, but extracting the right information is expensive due to several reasons: it takes time to go through publication databases and find relevant articles; the data is usually not in a directly usable format, but it needs organising and synthesising; the source of information is copyrighted, and it cannot be used as such without an explicit permission from the copyright owner, usually the journal. To decrease the costs of a risk assessment, relevant information should be systematically collected into a repository that is in public domain, i.e. the contents are freely usable by anyone. Risk assessors and researchers should be encouraged to provide the information that they have collected for their own assessments. Such a repository would benefit other assessors and the society at large. The extra work needed from information providers should be acknowledged as work for general good.

Specific methods related to Intarese general method

The specific methods are grouped below based on their relation to either:

  • making of an assessment (process); or
  • the contents of the assessment (product)

The methods related to the content of the assessment product are further divided under three categories:

  • methods related to the full chain
  • methods related to particular steps of the full chain; and
  • methods related to individual variables and indicators.

Process: making an assessment

Method Brief description
General properties of good risk assessments: Purpose of assessments Based on the purpose of risk assessment and the relations of risk assessment with its societal context, the general properties of risk assessment can be described in terms of effectiveness and efficiency. The general properties also function as the general performance criteria of risk assessment.
Causality: Full chain approach Risk assessments as causal network descriptions of the real-world phenomena from source to impact that are relevant in relation to the chosen endpoints and purpose of the assessment.
Collective structured learning Improving efficiency and effectiveness of assessments with commmonly applicable and re-usable, shared pieces of information and inter-assessment collaboration. Builds on the idea of a universal assessment structure.
Combination of value judgements and physical world descriptions Risk assessments should be able to explicitly scrutinize and combine value judgements of people with the descriptions of physical phenomena.
Collaborative work Carrying out risk assessments as sets of shared tasks in diverse groups for a common goal.
Managing stakeholder involvement
How and why to involve the stakeholders and decision-makers in the assessment work.
Assessment workspace
The assessment workspace is a virtual working environment that provides a platform for managing collaborative assessment work using Intarese general method.
Dealing with disputes How to deal with disputes that arise from the plurality of views, values and perceptions among the diverse group of contributors to the assessment.
Argumentation theory
external links: Argumentation theory | Pragma-dialectics
Argumentation provides the solution to dealing with disputes - no greater power than the power of a better argument.
Presentation of the assessment description How to present the contents of the assessment.

Product: contents of an assessment

Methods related to the full chain

Method/guidance name Brief description
Scoping and issue framing: Scoping tool for causal diagrams Defining the research questions, contents, and boundaries of the assessments.
Guidance and methods for indicator selection and specification
Selecting and specifying of the variables of specific interest across the causal network.
Creating a policy assessment protocol
Creating the plan for the assessment phase
Identification and evaluation of policy options How to identify, define, choose and apply different policy options within the assessment
Uncertainty and variability analysis | Value of information analysis | RIVM/MNP unicertainty guidance Understanding and addressing the different aspects of uncertainty and variability in the assessment. Value of information can be used for identifying the importance of different variables influence to the chosen indicators and decision options.
Probability theory and probabilistic methods Probability methods are used in many parts of the assessment. Basically all variable results are distribution estimates rather than point estimates.
Monte Carlo modelling
2D Monte Carlo for separation of variability and uncertainty
3D MC for separating variability, uncertainty, and ignorance
Monte carlo is a standard method for probabilistic simulations. It also provides means for simultaneosly and separately addressing different aspects of probability.
Bayesian analysis | Bayesian belief networks
External links: Bayes' theorem | Bayesian probability | Bayesian inference
Bayesian methods provide means for estimating probability distributions using heterogenous sources of data and subjective probabilities.
Decision analysis (external link)
Decision heuristics: utilitarian, egalitarian etc. | Risk aversion, risk premium
Ethical value systems
Decision analysis is a method for combining probabilities with preferences
The four indicator sets The four indicator panels are compilations of indicator sets taht can chosen for use according to the goals of the assessment. This approach is developed by RIVM.
1 Policy deficit indicators
E.g. exposures, concentrations, aims & goals, ...
2 Health impact indicators
E.g. number affected, severity, duration, ...
3 Economic consequence indicators
E.g. disease, absenteeism
4 Risk perception indicators
E.g. voluntariness, dread, ...
Alternative risk assessment approaches There are alternative approaches to describing the phenomena to be assessed. Many of these have similarities and overlaps with what are included in the Intarese general method.
DPSEEA model[1]
(Driving forces - Pressures - State - Exposure - Effects - Actions)
DPSIR [2]
(Driving forces - Pressures - States - Impacts - Responses)
Health impact assessment (HIA) | Review of health impact issues
Health impact assessment is a combination of procedures, methods and instruments used for assessing the potential health impacts.
Impact pathway approach (IPA)
Impact pathway approach is mainly based on estimating impacts and monetizing them.
MNP Environment and health planner
A risk assessment approach developed by MNP
MEME model
MEME: Multiple exposures, multiple effects.
Pyrkilo method |
Pyrkilo method is a risk assessment method which emphasizes the importance of openness and involvement of multiple parties throughout the assessment process.

Methods related to the steps of the full chain

Method/guidance name Brief description
Pressures Identification of pressures
Sources and Releases: Emission modelling Identification of sources, Emission factor approach, description how the stressor is released into the environment (e.g. emissions to air, leakage, dumping, corrosion)

Media: Environmental fate and transport modelling | Atmospheric modelling | Groundwater modelling | Multimedia modelling

Different media have different characteristics that need to be taken into account and that require different kinds of methods.
Exposure: Exposure modelling | Intake fraction modelling | Source-exposure modelling | Exposure-source attribution Models to estimate exposures given the emissions, and emission contributions given the exposures
Special exposure issues: Exposure mixtures | Multiple exposures, multiple routes
Dose: Physiologically based pharmacokinetic modelling (PBPK)
Dose metrics: Margin of exposure (MoE) | Tolerable daily intake (TDI) | Acceptable daily intake (ADI) | Hazard quotient (HQ) | No-observed adverse effect level (NOAEL)
Dose-response
Functions, shapes, and underlying assumptions (linearity, threshold, hormesis)
Theoretical issues related to the concept of dose-response.
Estimating dose-responses: Meta-analysis | Multibias modelling | Combining epidemiology and toxicology | Epitox workgroup
Meta-analysis is a method for combining data. Multibias modeling is a method to estimate the dose-reponse when there is external information about biases related to a published estimate. Methods on how to combine data from toxicological studies with data from epidemiological studies.
Extrapolation beyond data: Animal-to-human extrapolation | High-to-low-dose extrapolation | Endpoint extrapolation
How to use data on animal tests to estimating effects to humans? How to use data acquired from high-dose observations to low-dose situations? How to use data that is about a not exactly right endpoint?
Summary variables about dose-reponse: Reference dose (RfD) | Cancer slope factor (CSF)
Health effects and other impacts Health effects are the effects of the assessed phenomena to human health. Impacts include also other than health, such as monetary or ecological impacts.
Metrics for quantifying health effects: Life-table method | Attributable deaths and cases of illness
Issues related to health effects: Vulnerable groups and sensitivity
Some groups are more vulnerable because of higher exposure or because of higher sensitivity to a particular exposure.

Valuation

Combining different kinds of health outcomes and other impacts for comparison and/or summing up total impacts
Combining health effects: Quality-adjusted life years (QALY) (external link) | Disability-adjusted life years (DALY) | Years of life lost (YOLL)
QALYs and DALYs use slightly different weighting for health impacts.
Monetization: Indirect methods: Hedonic Price Method | Averting Behavior Method | Travel Cost Method | Contingent Behavior Method; Direct methods: Contingent Valuation Method | Attribute Based Choice Modelling | Participatory Approaches | Surveys for preferences of public decision makers; All these methods may ask for: Willingness-to-pay (WTP) | willingness-to-accept (WTA); Discounting | Cost effectiveness analysis and cost benefit analysis
There are several monetisation methods to combine all kinds of impacts into a single currency, money. Valuations should be applied coherently in one assessment and preferrably also between assessments

Actions: Developing decision options

Actions can be targeted to any part(s) of the causal chain and thus have influence(s) to a number of variables and indicators

Methods related to variables and indicators

Method/guidance name Brief description
Indicators and variables: Structure and attributes of variables Indicators are variables of specific interest. Describes the standard attribute list that can be used for all variables.
Model and function selection: Standardized set of validated methods Guidelines for evaluating and choosing models and functions for different purposes. Evaluated and recommendable methods to be used for desribing variables belonging to different steps in the causal chain.
Specific model use
Guidelines for using models in Intarese assessments
Data identification, evaluation, and selection Guidelines for evaluating and choosing data for different purposes
Data use and processing: Meta-analysis
Guidelines for using data in Intarese assessments. Meta-analysis is a method for combining results from several studies
Expert judgements
Method for eliciting subjective probabilities from experts using formal procedures.
Including stakeholder inputs: Having discussions about the content Stakeholder involvement as an method of collecting important data. Formalized discussions as a method of incorporating the knowledge of stakeholders to the assessment content

Related issues

  • Environmental health action plan of the EU
  • Expofacts
  • U.S.EPA Exposure handbook
  • FDA tolerance values
  • ATSDR toxicological profiles
  • HHS/NTP substance profiles
  • NIOSH current intelligence bulletins and criteria documents
  • MEME appraoch
  • WHO ECOEHIS indicators
  • As low as reasonably achievable (Alara)
  • Tools needed in Intarese toolbox


Properties of a good risk assessment method. An old version (archived).
Template:Release

An 'axiomatic theory' of environmental health risk assessment

Axiom 1

Risk is such a property of a situation that tells that a) the situation may lead to an unfavourable outcome; b) it is uncertain which outcome actually will occur; and c) there is a need to affect which outcome will occur.

Axiom 2

Risk assessment is a process of developing a rational and reasonable description of a particular risk so that more undestanding is achieved about how to reduce the need to affect the outcome.

Axiom 3

A risk assessment consists of statements that are either variables (statements about particular properties of the world, including both physical properties (what is?) and value judgements (what should be?)) or links (statements about the relationships between variables). Each statement has a focus and a scope.

Axiom 4

Risk assessment science is a discipline that answers to the following question: "When should I, as a rational critic who judges reasonably, regard a risk assessment as acceptable?"

Axiom 5

A statement in risk assessment is regarded validated iif it holds against critical argumentation about the content and relevance (according to van Eemeren and Grootendorst). The argumentation must be based on accepted premises and discussion rules. Falsification (according to Popper) is a special case of argumentation, where the validation is based on the consistence between the statement and existing data. The validation process and its result are parts of the statement.

Axiom 6

The axiom of efficiency: A new variable must not be created, if there already exists a variable with the same focus and scope.

Axiom 7

The scientific statements, value judgements, and validation rules in a risk assessment must all be internally coherent and consistent. If the assessment contains conflicting statements, there must be a rule for determining which of the statements holds in a particular situation.


Rules derived from axioms:

  • Based on axioms 3 and 6: Focus and scope of a statement must not change during the risk assessment. Statements may be structured hierarchically.
  • Based on axiom 7: A risk assessment based on one or more invalid statements is invalid.
  • Based on axiom 5: To be able to compare risk assessments, they must be based on cohorent premises and discussion rules. Thus, there is a need to develop systematic sets of standardized premises and discussion rules that will be used.
  • Based on axiom 6: The focus and scope of a statement must not change. This is because a statement may be used in several assessments, and the statement may lose its valid content or relevance, if its focus and content are changed.
  • Based on axiom 5: Validation process of a statement may change the content (but not the scope and focus) of the statement. This is allowed. However, the risk assessor must be explicit about which version of the statement he refers to, otherwise it will be difficult to follow the reasoning.
  • Based on axiom 7: Examples of inconsistent statements include the following:
    • Probability distributions: Only one value is possible in each particular situation, and random sampling is the rule how to determine which one. Correlation matrices are also this kind of rules.
    • Disagreement variables express disagreement about values. The rule may be e.g. that the risk assessor chooses the one he thinks relevant, or that all values are evaluated one at a time.
    • Dispute variables express dispute about scientific statement. What is said about disagreement variables, applies also here.

Estates of risk assessment

The philosophical estate:

Q: When should I, as a rational critic who judges reasonably, regard a risk assessment as acceptable?

A: When the risk assessment solves the difference of opinion in accordance with "problem valid" assessment rules (solving the problem at issue) that are also acceptable to the parties.

Onko RA samanlainen kahden osapuolen väittely kuin argumentaatio? Kuka yritetään vakuuttaa? Ei se ole, vaan se on enemmän kuin Toulminin ja Perelmanin näkemys puhujasta, joka yrittää vakuuttaa skeptisen kuulijakunnan (joka ei osallistu aktiivisesti). Kuitenkin voidaan ajatella, että on olemassa jokin kysymys, ja siihen liittyvä haaste jonka todenperäisyys täytyy selvittää. Kun nämä molemmat on kuvattu, se määrää pitkälle sen, kuinka yksityiskohtaisen täytyy riskinarvioinnin olla, jotta selvyys kysymykseen saadaan.

Se, että joissakin tilanteissa kysyjä ja vastaaja on sama (kuten huippuyksikössä) ei muuta tilannetta. Toisaalta, riskinarviointi on jatkuva prosessi, jossa lisäkysymyksiä voi tulla matkan varralle kun ensimmäisen kierroksen tulos julkaistaan. Kuitenkin täytyy olla sääntö siitä, milloin riskin arviointi päättyy ja alkaa uusi arviointi. Yleissääntö kai on se, että kun alkuperäiseen kysymykseen on vastattu, arviointi päättyy, ja lisäkysymykset ovat lisäarvioinnin paikka.

Tieteen voi ajatella jakautuvan seuraavasti:
  1. Kokeellinen tiede tekee havaintoja koeasetelmista, joissa tuntemattomat tekijät on pyritty satunnaistamaan pois.
  2. Havainnoiva tiede tekee havaintoja perustuen tiettyyn menetelmään ja tutkimusasetelmaan jolla pyritään hallitsemaan tuntemattomia tekijöitä.
  3. Mallitus perustuu ennustuksiin, joiden toteutumista arvioidaan (validoidaan) havainnoin.
  4. Riskinarviointi perustuu ennustuksiin, joiden toteutumista ei voida kaikilta osin arvioida havainnoin. Tällöin ainoaksi validoinnin keinoksi jää koko tarkastelun ja ennusteiden alistaminen vapaaseen kriittiseen arviointiin. Arvioinnista selviytyminen ei todista ennustetta oikeaksi mutta se lisää todennäköisyyttä että väärät on osoitettu vääriksi prosessin aikana. → Tässä on nimenomaan kyse argumentaatioprosessista.

The theoretical estate:

Q: Which instruments are available to me for treating problems concerning the acceptability of a risk assessment systematically?

A: I can make use of an ideal model of risk assessment aimed at describing a risk and a series of rules for the assessment procedure that are relevant.

Millainen on RAn ideaalimalli? Rajataan aihe, määritellään tarkoitus (tarkasteltava päätös), valitaan lopputulos, kuvataan syy-seuraussuhteet ja tehdään VOI. Tämä asetetaan yleisesti arvioitavaksi.

Voidaanko luoda yleinen sääntökokoelma, jota täytyy noudattaa? Käytännössä tarvittaisiin useampia kokoelmia, joista täytyy valita (tai joukko asioita, joista pitää laatia säännöt soveltamisesta).Olisi hyvä olla jokin perustuslaki, joka ainakin täytyy täyttää. Tuo yllä mainittu voisi toimia semmoisena.


The analytical estate

Q: How can I obtain a clearer picture of everything that is relevant for my evaluation of a risk assessment?

A: By reconstructions the risk assessment as an attempt to resolve a difference of opinion by conducting the required dialectical transformations.

Tässä voisi ajatella syy-seuraussuhteiden nostamista ylätasolle, ja yksityiskohtien painamista alatasoille. Olisiko tämä toimiva systeemi muuntaa riskinarviointi noudattamaan oikeaa formaattia? Voisi myös edellyttää, että riskinarvointi aluperinkin laaditaan käyttäen dialektisia sääntöjä. Toisaalta, on hyvä olla mahdollisuus siihen, että arviointi tehdään alunperin helpolla tyylillä käyttäen luonnollista kieltä, ja vasta myöhemmässä tarkastelussa se muutetaan noudattamaan formaalisia sääntöjä.

The empirical estate:

Q: Which knowledge about risk assessment reality that may be of special use to me can I acquire?

A: I can investigate which factors and processes are important in risk assessments to convince someone who is in doubt of the acceptability of an assessment.

Tämä on periaatteessa helppo: tarkastellaan oikeita riskinarviointeja ja arvioidaan niiden sisältöä kriittisesti ideaalimallin näkökulmasta. Katsotaan myös sitä, miten eri riskinarvioinnit on otettu vastaan.

The practical estate:

Q: How can I contribute to the improvement of risk assessment practice?

A: I can promote reflection on the procedures that are used in different risk assessment practices and the skills that are required for an adequate production, analysis, and evaluation of risk assessment practices.

The texts below are from van Eemeren, Grootendorst: A systematic theory of argumentation (Cambridge University Press, Cambridge, 2004). The texts about risk assessment are created by analogy from the texts in the book. Some texts are shortened.

A systematic theory of argumentation

The philosophical estate:

Q: When should I, as a rational critic who judges reasonably, regard argumentation as acceptable?

A: When the argumentation solves the difference of opinion in accordance with "problem valid" discussion rules (solving the problem at issue) that are also acceptable to the parties.

The theoretical estate:

Q: Which instruments are available to me for treating problems concerning the acceptability of argumentation systematically?

A: I can make use of an ideal model of critical discussion aimed at solving a difference of opinion and a series of rules for the performance of speech acts that are relevant in such a discussion.

The analytical estate

Q: How can I obtain a clearer picture of everything that is relevant for my evaluation of an argumentative discourse or text?

A: By reconstructions the discourse or text as an attempt to resolve a difference of opinion by conducting the required dialectical transformations.

The empirical estate:

Q: Which knowledge about argumentative reality that may be of special use to me can I acquire?

A: I can investigate which factors and processes are important in argumentative discourse to convince someone who is in doubt of the acceptability of a standpoint.

The practical estate:

Q: How can I contribute to the improvement of argumentative practice?

A: I can promote reflection on the procedures that are used in different argumentative practices and the skills that are required for an adequate production, analysis, and evaluation of argumentative discourse.

Rules for argumentation

Rule 1

The difference of opinion may concern any standpoint. Any discussant may call into question any standpoint.

Rule 2

If someone has called a standpoint into question, he is entitled to challenge this discussant to defent his standpoint in the confrontation stage.

Rule 3

A discussant is obliged to defent his standpoint, if he and the antagonist agree on the premises and discussion rules.

Rule 4

The discussant who originally presented the standpoint is a protagonist, and the discussant who called this into question is an antagonist, unless they decide otherwise. The roles are maintaind until the end of discussion.

Rule 5

The protagonist and antagonist must agree on discussion rules and criteria on when the standpoint has successfully been defended or attacked.

Rule 6

The protagonist may defend the standpoint using argumentation. The antagonist may attack the standpoint by calling into question either the propositional content or the justificatory force of the argumentation. Other acts are not allowed.

Rule 7

Intersubjective identification procedure is used to determine if the argument has been successfully defended or attacked.

Rule 8

Intersubjective inference procedure is used to determine if the force of justification or refutation of the argumentation has been successfully defended or attacked.

Rule 9

An initial standpoint has been conclusively defended (attacked) if both (either) the propositional content and (or) the force of justification of the argumentation has been successfully defended (attacked).

Rule 10

The antagonist may at any time during the discussion call into question any argumentation that has not yet been successfully defended.

Rule 11

The protagonist may at any time during the discussion defend any argumentation that he has performed and not yet successfully defended against every attack by the antagonist.

Rule 12

The protagonist retains throughout the entire discussion the right to retract any argumentation that he has performed, and thereby to remove the obligation to defend it.

Rule 13

a) The protagonist and the antagonist may perform the same speech act with the same role in the discussion only once.
b) The protagonist and the antagonist must in turn make a move of (complex) speech acts with a particular role in the discussion.
c) The protagonist and the antagonist may not perform more than one move (complex) speech acts at one time.

Rule 14

Standpoints that have been cuccessfully attacked must be retracted, and those successfully defended must no longer be called into question. All other standpoints are open to defence and critique.

Rule 15

Clarification may be asked at any stage of the discussion, and clarification must be given when asked for.

Ten commandments for reasonable discussants

  1. Discussants may not prevent each other from advancing standpoints or from calling standpoints into question.
  2. Discussants who advance a standpoint may not refuse to defend this standpoint when requested to do so.
  3. Attacks on standpoints may not bear on a standpoint that has not actually been put forward by the other party.
  4. Standpoints may not be defended by non-argumentation or argumentation that is not relevant to the standpoint.
  5. Discussants may not falsely attribute unexpressed premises to the other party, nor disown responsibility for their own unexpressed premises.
  6. Discussants may not falsely present something as an accepted starting point or falsely deny that something is an accepted starting point.
  7. Reasoning that an argumentation is presented as formally conclusive may not be invalid in a logical sense.
  8. Standpoints may not be regarded as conclusively defended by argumentation that is not presented as based on formally conclusive reasoning if the defense does not take place by means of appropriate argument schemes that are applied correctly.
  9. Inconclusive defences of standpoints may not lead to maintaining these standpoints, and conclusive defenses of standpoints may not lead to maintaining expressions of doubt concerning these standpoints.
  10. Discussants may not use any formulations that are insufficiently clear or confusingly ambiguous, and they may not deliberately misinterpret the other party's formulations.


Toolbox is a software for applying the Pyrkilo method. Details of is are shown on the graph below. Click on a detail to get more specific information.

  1. Collaborative workspace: user interface for the risk assessors: describing risk assessments and variables, using the principles of mass collaboration (a large group of interested people working together for a common goal) and argumentation (rules for resolving disputes using structured argumentation). The collaborative workspace contains four kinds of information:
    • Openly available and usable, structured information. This is mainly in the form of variables.
    • Openly available and usable, but freely structured information. The major part of this content is in the form of an encyclopedia and glossary, but there are also fact sheets, guidance documents, and data and models there.
    • There can also be structured information as variables that have a limited access. This may be the case in some so called private risk assessments, where the needs of confidentiality are stronger than the benefits of mass collaboration.
    • There can be metadata about the data existing in external sources, i.e. data that is not openly available and usable and is not structured as variables. This metadata is also called the Data gateway.
  2. Separate calculation tools: a functionality to transform variable definitions into a model that can compute results using an external software, such as Analytica or R.
  3. External models: If a variable requires complex computing (such as atmospheric modelling), the actual definition can be externalized to a collaborating institute. The institute runs the model, and only the result is made available in the result database.
  4. Result database: contains previously calculated results of variables, so that they are available for risk assessments and for calculating downstream variables.
  5. Analysis tool: This tool performs the assessment-specific analyses, such as scenario optimising, sensitivity analyses, or value-of-information analyses. (Assessment-specific means that the analysis is not a part of any single variable but is an analysis about a group of variables within an assessment.) These analyses are based on the result database.


Help:Variable transfer protocolHelp:Variable transfer protocolHelp:Variable transfer protocolHelp:VariableHelp:ClassHelp:Purpose of a risk assessmentHelp:Purpose of a risk assessmentHelp:Purpose of a risk assessmentHelp:Purpose of a risk assessmentHelp:Applying general informationHelp:Data gatewayHelp:Collaborative workspaceHelp:Risk assessment structureHelp:Scoping a risk assessmentHelp:Drawing a causal diagramHelp:Designing variablesHelp:Executing variablesHelp:Executing variablesHelp:Risk assessment structureHelp:Result distribution databaseHelp:Variable transfer protocolHelp:Analysis toolHelp:EncyclopediaHelp:Collaborative workspaceHelp:Collaborative workspaceRisk assessment method - process.PNG
About this image

Use this template instead of the image:Risk assessment method - process.PNG so you will get direct links to the help pages of the different parts of toolbox.


This document is produced for Intarese Subproject 3 in the startup phase. It contains guidance from Subprojects 1 (and 4?) to facilitate the policy assessment drafting. The deadline for this document is mid-May. About a page per topic.

Assessment framework

  • Define draft focus of the assessment. Is it driven by e.g. a decision at hand, or a particular outcome whose causes are assessed?
  • Define draft scope of the assessment. What is to be assessed and what is to be excluded from the assessment? Describe the value judgements that you use to defend your selection.
  • Define draft indicators that will be looked at, and select one as the main output indicator. The indicator list should always contain indicators from each of the following groups (or reasoning why a group was ignored).
    • Policy target indicators (how far is the target)
    • Impact indicators (health, cost)
    • Public acceptability indicators (equity, perception)
    • Appraisal indicators (cost-benefit analysis, multicriteria analysis)
  • Define draft links between the indicators and the focus within the scope. All variables must link to the focus directly or indirectly.
  • Remove everything that is not linked to the causal chain of the assessment, because it is irrelevant.
  • Describe the value judgements that relate to the causal chain of the assessment.
  • Present the draft assessment structure to the stakeholders.
  • Based on the deliberation, revise the assessment as needed.
  • Finalise the focus and the scope. Do not change them any more within this assessment.
  • Add and improve statements (variables and links) that are relevant, i.e. linked to the focus and within the scope.
  • Let stakeholders criticise your work using the discussion rules of critical argumentation. Stakeholders may also be imaginary.
  • If any statement is not convincing for a rational and reasonable critique, add more argumentation to defend it. If unsuccessful, invalidate the statement. (Invalidating means that they are labeled invalid and they are not used for deduction; however they must stay parts of the assessment anyway to show that they cannot be used.)
  • Stakeholders may add statements to the assessment. These must be added unless they can be invalidated.
  • When no further critique is presented, label the risk assessment as finished (until additional argumentation is presented).

A risk assessment is always conditional to the data existing at the time. (In MediaWiki environment this means that when a referred page changes, the risk assessment is based on a non-current version. To avoid confusion, permanent links should be used so that links always point to the version that actually was used to deduce the conclustions of the assessment).

Source-exposure

Exposure-response

Risk characterisation

  • Describe the focus of the assessment and reasons for selecting this focus.
  • Describe the scope of the assessment and reasons for selecting this scope.
  • Describe the main assumptions used in the assessment.
  • Describe the indicators of the assessment and reasons for selecting these indicators.
  • Describe the dimensions and properties of the indicators. Does it represent average, a random individual, or certain fractile? Describe uncertainty bounds around the estimate. Especially describe the different kinds of indicators
  • If possible, compare the importance of each uncertain variable to the output indicator. (e.g. value-of-information analysis or importance analysis).
  • Describe the main conclusion derived from the output indicator in respect to the focus.
  • Describe other conclusions.


Further reading related to risk characterisation

  • DPSEEA http://www.euro.who.int/EHindicators/Indicators/20030527_2
  • DPSIR http://www.eea.europa.eu/documents/brochure/brochure_reason.html
  • Retrieved from "http://opasnet.org/en-opwiki/index.php?title=Pyrkilo&oldid=13920"