Difference between revisions of "Wikimania 2009 poster abstract: Opasnet - a wiki site for improving societal decision-making"

Revision as of 10:33, 24 July 2008

Poster abstract for Wikisym 2008 will be written on this page

Introduction

Open Assessment (OA) is an ontology-based approach to assessing real-world phenomena and communicating the assessment results to those who need or wish to use them. It consists of a conceptual method that defines both the assessment process and its products, and an information system to support the application of the method. It is currently being developed and applied within the domain of environmental health assessment, but the usage of the method is not limited to any specific domain. Open Assessment has been developed by the National Public Health Institute (KTL) in Finland.

As its name implies, Open Assessment relies on open participation and freely available content which makes Mediawiki an essential tool for applying this method. By using Wiki in OA we aim to:

1. provide information to public audience
2. give public audience a chance to discuss about and contribute to assessments
3. collect information from public audience

We believe that by means of mass collaboration also the quality of content in the assessments will increase. It also increases the level of transparency, since, in an ideal case, all information related to a particular assessment is freely accessible in the internet and open to contributions as soon as an assesment has been started. In some cases there may be limitations to openness, e.g. if confidential data is being used in assessments. In such cases the level of openness just needs to be limited and only those parts of the assessment that are not confidential are set openly available.

Information structure

In order to make open participation to assessing complex real-world phenomena possible, a clear and systematic information structure for representing information and targeting contributions is required.

The basis of the information structure of Open Assessment is a variable. It is an independent object (chunk of information about reality), given its relations to other relevant objects. An Open Assessment may consist of tens or even hundreds of variables. For example, a variable called Primary PM2.5 emissions from bus traffic in Helsinki Metropolitan Area is a part of an assessment (possibly among others) called Gasbus - health impacts of Helsinki bus traffic and describes the total amount of fine particulate matter (PM2.5) emitted by buses in Helsinki, Finland.

All variables share the same basic structure with 4 main attributes:

• Name

Name attribute is the identifier of the variable, which also defines what real-world entity is considered in the variable. Name should be descriptive and unique so that two variables cannot have the same name.

• Scope

Defines the boundaries of variable - what is included and what is excluded. Boundaries can relate e.g. to time and space, but can also be abstract. For example a variable may be limited to cover only year 2007 when all the other years are excluded from consideration. In principle, scope explains what is the question about the reality that the variable attempts to answer.

• Definition

Explains how the question determined in the scope is answered to. Definition is divided into 4 sub-attributes:

• Causality

Defines what other variables affect (are causes of) this variable and how.

• Data

Describes what kind of data is used in this variable (e.g. measurements or expert judgements). It may also contain links to available datafiles.

• Unit

Defines what unit is being used in presenting the result of this variable.

• Formula

Describes the formula used to calculate the variable result (if calculable).

• Result

Result of the variable, most preferable a probability distribution. Result can also be non-numerical expression.

In Open Assessment variables are open for argumentation and outcomes of disputes may change the actual content of the variables and by this way affect the result of the assessment(s) it is a part of. We believe that by means of open participation we can obtain more data and take account of more points of view in variables and thereby achieve better results. Anyhow, we also believe that even without any public participation the information structure helps to improve the transparency of assessments compared to other approaches where most often only the final results of the assessments are published.

Technical facilitation

Open Assessment method imposes a few challenges to technical facilitation of applying the method. We needed to have a flexible system which enables mass collaboration and we also needed advanced computational capabilities with Monte Carlo analysis features. Currently we are building up a web-based system called Opasnet (Open Assessor's Network). Opasnet will contain guidance how to apply OA method in practice and also provide a free platform for carrying out Open Assessments.

We chose Mediawiki as the main software for Opasnet because it allows to describe variables in mass collaboration and efficiently communicate the results. Mediawiki is also widely used and has a good list of different extensions available.

Mediawiki is not designed for mathematical modelling so we needed another software for that purpose. We chose Lumina's Analytica which is a visual tool for creating, analyzing, and communicating decision models including Monte Carlo analysis. We have built some linkages between Analytica and Mediawiki. Analytica also has a free player-version available. Models are uploaded into Wiki database and they can also be launched directly from wiki (if user has at least the player version of Analytica installed).

Latest addition to the OA information system is the Result Database. It is basically a MySQL-database which is used to save the results of variables. This is needed because neither Mediawiki or Analytica is capable of storing huge amounts of numerical data. Also it is not always a good idea to re-run models without significant changes in their parameters because larger model runs can take even days to complete. Therefore we need Result Database from where needed results can be fetched quickly. Results are uploaded into the Result Database directly from Analytica. From Result Database results can be fetched directly into corresponding Variable-page in Mediawiki. Variable-page can also contain link to Result Database's user-interface where more detailed results can be provided. It will also be possible to download results as a csv-file. The Result Database is currently in test use and for the time being only few persons are capable/allowed to use its functionalities to full extent.

Future challenges and enhancements

One of the biggest problems in applying OA method is that most scientific journals refuse to publish research results that have already been published in the Internet. This brings about limitations of what can or can not be made publicly available and when. This is quite a tricky question because many contributors to Open Assessments may also need or want academic credits for their work.

Also the use of wiki syntax seem to be an obstacle for some potential users. We are currently trying to fix this by lowering the threshold for using wiki with the aid of FCKeditor but there are still some issues to be solved with the functionalities of the editor.

At some point the growing Result Database will probably lead to increasing demands from the database server. This challenge will be addressed by updating our platform from virtual server to a dedicated server in the near future.

Poster

Poster will describe the basics of OA method. It will also explain whole OA system's technical solutions, their linkages and system architechture.