Difference between revisions of "Wikimania 2009 poster abstract: Opasnet - a wiki site for improving societal decision-making"

Revision as of 07:06, 6 June 2008

Poster abstract for Wikisym 2008 will be written on this page

Introduction to Open Assessment

Open Assessment method (OA) is a assessment method developed by National Public Health Institute in Finland. Open Assessment relies on open participation and free content which makes Mediawiki essential tool for this method. By using Wiki we aim to:

1. give public audience a chance to discuss about and contribute to assessments and decision making
2. provide information to public audience
3. collect information from public audience

We believe that by including mass collaboration into assessments the quality will increase as well. This will also increase the level of transparency because in ideal case everything related to a particular assessment is freely available from internet and open for contributions at the same time someone starts the case. Unfortunately there can be some limitations if some confidential data is being used in assessments. In that kind of cases we just need to limit the level of openness and publish what is possible.

Single most important part of Open assessment method is a variable. In OA method assessments consists tens or even hundreds of variables. Each variable describes specific part of some real-world situation. For example variable Primary PM2.5 emissions from bus traffic in Helsinki Metropolitan Area would describe PM2.5 emissions in Helsinki Finland.

All variables share the same basic structure with 4 main attributes:

• Name

Name attribute is the identifier of the variable, which more or less describes what the real-world entity the variable is. Name should be descriptive and unique so two variables cannot have same name.

• Scope

Defines boundaries of variable - what it describes and what is excluded. Boundaries can relate e.g to time and space. For example time can be limited to cover only year 2007 when all the other years are excluded from variable.

• Definition

Definition is divided into 4 sub-attributes:

• Causality

Defines what upstream variables affects this variable and how.

• Data

Describes what kind of data we have about this variable (e.g. measured or expert judgements). May also contain links to available datafiles.

• Unit

Defines what unit is being used in the result of this variable.

• Formula

Describes the formula used to count variables result.

• Result

Result of the variable, most preferable a probability distribution. Result can also be non-numerical expression.

In OA method variables are open for argumentation and dispute results may change variable's actual content and by this way affect the whole assessment case result. We hope that by using OA method we can obtain more data and point-of-views about variables and therefore also reach to a better results. Even without any public participation transparency of assessments is increased compared to old fashioned assessments where only the results are published.

Technical solutions

OA method sets a few challenges to software requirements. We needed to have a flexible system which enables mass collaboration and we also needed advanced computational capabilities with Monte Carlo analysis features. Currently we are building up a web-based system called Opasnet (Open Assessor's Network). Opasnet will contain guidance how to apply open assessment method in practice and will also provide a free platform for implementing open assessments.

We chose Mediawiki as main software for Opasnet because it is widely used and it has good list of different extensions available. In OA method Mediawiki site is used to describe variables, cases and data as well it also enables mass collaboration.

Mediawiki is not designed for mathematical modelling so we needed another software for that purpose. We chose Lumina's Analytica which is a visual tool for creating, analyzing, and communicating decision models including Monte Carlo analysis. Analytica also has a free player-version available. We have been building some linkage between Analytica and Mediawiki. Models are uploaded into Wiki database and they can be opened directly from there (if user have at least player version of Analytica installed).

Latest addition to OA software family is Result Database. It is basically a MySQL-database which is used to save results of variables. This is needed because neither Mediawiki or Analytica is capable of storing huge amount of numerical data. Also it is not always a good idea to re-run models with same parameters because larger models can take even days to complete. Therefore we need Result Database from where results are quick to fetch. Results are uploaded into Result Database directly from Analytica. At the moment only few persons are capable/allowed to do this. From Result Database results can be fetched directly into corresponding Variable-page in Mediawiki. Variable-page can also contain link to Result Database's user-interface where more detailed results can be provided. It will also be possible to download results as csv-file.

Challenges and future enhancements

One of the biggest problems in OA method is that most scientific magazines do not accept articles or results which has already been published in Internet. This is quite tricky question because many contributors need also academic credits for their work.

Wiki syntax seem to be an obstacle for some potential users. We currently trying to fix this with FCKeditor but still there is some issues to be solved.

Growing Result database will at some point lead to increasing demands from server. Therefore we are planning on updating our platform from virtual server to a dedicated server in the near future.

Poster

Poster will describe the basics of OA method. It will also explain whole OA system's technical solutions, their linkages and system architechture.