Implementation plan of evidence-based policy

This page is a knowledge crystal of subtype variable. The page identifier is Op_en7391
Moderator:Jouni (see all)
Upload data Show results

Scope

How can the better policy practises identified in Yhtäköyttä-project be implemented into national governance? Especially actions that support the government are examined.

Answer

The implementation plan includes three new social innovations that will be taken into use in the decision support for the government's work. These are a) shared understanding as a goal for knowledge work, b) variables aka knowledge crystals in structuring and interpreting of information and c) respect theory to motivate the production of information.

The implementation plan is built on top of open assessment and open policy practice, and you should also read descriptions of those methods.

Shared understanding as a goal for knowledge work

<section begin=glossary />

Shared understanding between two individuals about a particular topic means a situation where each individual is able to correctly explain what the other thinks about the topic and why. This definition can be extended also to a larger group. Here shared understanding is a written description of the topic that covers the thinking and reasoning on the topic of all members. In this group setting, not everyone needs to be able to describe everyone else's thinking, but everyone should agree that the written description correctly contains their thinking about the topic. In this way, although not everything is known by everyone in the group, the written description effectively represents the shared understanding of the group. Importantly, online tools such as wikis can be used to develop shared understanding even about complex topics among large groups.

It is important to notice that shared understanding is not the same as consensus or agreement. In shared understanding people can still disagree on the topic, but they agree on what opinions there are about it. While the purpose of agreement in decision-making is to conclude about the best option and act based on it, the purpose of shared understanding is to identify one or more poor decision option that should be rejected. This is analogous to the scientific method. Instead of attempting to find the truth directly, it critically evaluates hypotheses and rejects all that are not plausible in the light of observations. As the scientific method has proved to be the best method known for developing knowledge about the world, shared understanding may prove to be an effective way to make public policy.

<section end=glossary />

Shared understanding in the work of the government

In decision support using shared understanding, for example in drafting a bill, it is not enough that a law text is created with valid arguments. It must also be described what other points of views there were and what were their arguments. Even views that were once suggested seriously but that turned out to be invalid will be described. This is important so they don't resurface when people not familiar with the previous work join the project. This could also be important for getting the acceptance of the public and others who have not participated.

To promote shared understanding as a practice, the law preparations will need a new, open web-workspace. It can be fully open (in the internet) or, if the situation requires, only open to those participating in the preparations. From the point of view of trying out the method, completely open preparation would be the most educational.

Additionally the work must be done in shared information objects. All separate topics have their own pages in the workspace, and all connected material is gathered on the page no matter who produced the information. The research question for each topic is precisely defined, and the purpose of the work is to produce the best possible answer to it. This means a transfer from a person-based information structure to a topic-based information structure. This way everyone participating in the preparation will have at hand the same information and tools, and are capable of evaluating what information is still missing from the goal of shared understanding. The schedule and to-do-list for the project will also be open, so following the case is easier. Separate to-do-lists are avoided, because they make it harder for the information to spread.

Thus the goal of shared understanding creates the following action proposition:

• Action 1. Shared understanding is taken as a spelled out goal in projects that want to use the implementation plan.
• Action 2. The preparations for the projects are done in an open web-workspace. The existing tools can still be used, but all information will be brought to the workspace without delay.
• Action 3. The information is structured in the workspace by topic. Research questions for each topic are defined.
• Action 4. The person in charge of the project opens the schedule and to-do-list of the project for everyone to see and maintains it for the durations of the project.

Knowledge crystals to structure the interpretation

A knowledge crystal is a web page that aims to answer a specific question by using principles of open science – notably open participation, criticism, and permanent resource locations. Each knowledge crystal aims at finding a good answer or answers with rationale that convinces a critical reader. Arguments and rationale can build on open data, references to scientific research, discussions, calculations or models, or basically anything useful. The concept was originally developed in the National Institute for Health and Welfare (THL).

Naturally, these open web pages with permanent locations offer a convenient way of checking the current state of understanding about each topic for scientists, policy-makers, journalists and curious citizens alike. This way the project furthers deliberate and structured societal discourse.

For a presentations about their use, see Online collaborative models. Knowledge crystals are extensively used in Opasnet, where they are mainly in forms of variables, assessments, and methods. For descriptions of recent use of knowledge crystals, see Portal:Variables.

Kansalaiskide (Citizen Crystal) project aimed at reshaping information flows and knowledge creation in society by developing methods for the open co-creation of knowledge. Kansalaiskide was one of the five finalists of Uutisraivaaja contest in 2019. Uutisraivaaja is a media innovation contest organized by the Helsingin Sanomat Foundation. The contest seeks ideas for developing media, journalism and the distribution of information at large.

Knowledge crystals also developed in an open source project of Open Knowledge Finland ry funded by various public and private sources.

Question

What do knowledge crystals have to be like to

• be useful information odjects in impact assessments as they are,
• contain the answer as open data,
• withstand scientific critique,
• be able to measure the use and usefulness of the knowledge they contain,
• be able to, in an acceptable way, hand out scientific merit to the people involved in producing the content?

Answer

Knowledge crystals are the basic elements of for example assessments. They always describe a phenomenon of the real world. These can be the descriptions of physical phenomena, like exposure to a chemical, but also for example the population's opinion distribution on immigration. It is in the nature of knowledge crystals they are not final, but their content develops with new information and work put into them. Knowledge crystals are also not tied to any specific assessment, but can be used as parts of multiple assessments. An exception are assessments, that are produced to help with a certain decision, and whose answer doesn't change after the assessment is finished (even though the variables in the assessment may change). Knowledge crystals are also called variables because that's the role they have in assessment models. However, the word variable has so many other meanings that we prefer knowledge crystals in this context.

Another basic feature of a knowledge crystal is its standardised structure that enables the building of assessment models or different internet applications basing on it. So even though the content is updated as knowledge increases, a knowledge crystal remains in the same, computer-readable format. Usually only raw data is in more or less standard format, while the information object containing interpretations from the data are almost without exception made for humans instead of computers, like articles or reports. This makes the knowledge crystal a rare kind of information object: it is computer-readable interpretation of some specific topic.

There are different kind of knowledge crystals for different uses, and they are more accurately described on for example the pages variable, assessment and method. Here is a short description of the most important qualities of a knowledge crystal.

• Knowledge crystals answer a specific research question.
• The answer of a knowledge crystal is the current best synthesis of all available data. Typically it has a descriptive easy-to-read summary and a detailed quantitative result published as open data. An answer may contain several competing hypotheses, if they hold against scientific criticism. This means it also includes an accurate description of the uncertainty of the answer.
• The rationale of knowledge crystals includes all information that is required to convince a critical rational observer of the validity of the answer.
• The content of knowledge crystals is produced by crowdsourcing. Anyone can participate.
• Knowledge crystals are aiming to find shared understanding. It is a situation, where all participants' views have been described well enough so that people can know fact facts and opinions exist about the topic and what agreements and disagreements exist and why.

Rationale

Different information objects and their usage

Knowledge crystals contain scientific knowledge, but they differ from classic products of scientific research. Here is a short description and comparison.

A scientific article is the basic unit of publishing science today. For it a researcher or a research group produces data, i.e. observations about the world. The data is analysed, and in the end interpretations and conclusions are made based on the new results and previous scientific articles. The goal is to publish the article in a peer reviewed journal. Peer review means that a few researches in the field look through the manuscript and back it up before it is published. The peer review system aims to raise the quality of the manuscripts and weed out bad research. It is commonly agreed that the system isn't especially efficient for either purpose, but no one has come up with anything better. Someone has said that the primary product should be the original data, not an article: researchers should publish what they found, instead of writing descriptions about what they think they found.

Expert reports are gathered by an expert well familiar with the field in question, and are usually about some specific question like the topic of a future decision. They produce new knowledge but not new data. They are usually not peer reviewed, so they're not well respected among researchers and research funders. However, they are much better suited for decision support, because they answer the actual questions that are relevant to the decision at hand.

Open data is usually measured raw data that has been made public for anyone to use. It depends on the case whether the data is well cultured and quality-proofed, but it often has quality issues such as poor meta data. The practises of open data have only begun to take shape in the last few years, because researches haven't been in the habit of publishing raw data before. The problem with supporting decision-making with raw data is that it doesn't involve any interpretations or conclusions, and even less so of the relevant issues. Open data is great raw material for someone who knows how to analyse and interpret it and has the time, but quite useless to anyone else.

The idea of a knowledge crystal is to combine the parts of other information products useful to decision support and avoid the bad parts. The idea of a knowledge crystal is to build an information object around a specific research question. The question can be purely scientific, but in the case of decision support it is usually phrased to help precisely the future decision. To answer the question experts gather all possible material that will help answer the question. This includes research articles, expert reports, open data and all other silent knowledge of the experts that is not found in written form.

The knowledge crystal is worked on from the beginning in an open web-workspace with the help of crowdsourcing, and all information it contains is free to use. The material is structured, assessed and interpreted. The result is an answer that has passed all critique that has come up during the working process. Thus the answer is the best current interpretation of how the thing the question asks is in reality. Criticising the knowledge crystals openly during the work ensures that the answer is scientifically sound. The answer is usually in a computer-readable format for models to use and also in text and picture format for humans.

The strengths of a knowledge crystal are that it uses all relevant information (not only own data as in an article), interprets the data (unlike open data) and is produced by following the principles of openness and critique (unlike an expert report).

Producing shared understanding by utilising knowledge crystals

Main article: Shared understanding

A key objective of strategic research is to support societal decision making. This should be done already from the beginning by utilising a method called open policy practice. It was developed in THL in 2013 and it is based on long-term experience on decision support in environmental health. The most important principle of open policy practice is to develop shared understanding about a policy issue at hand. Shared understanding is a situation, where all participants have collaboratively described in writing what is known about the details of the issue, what are objectives of different stakeholders, where there are agreements and where there are disagreements and why. Participation is open and includes decision makers, experts, citizens, and other interested parties.

Shared understanding is reached by utilising systematic methods of collaborative work and participation. When there is disagreement about facts, resolution is found by using criticism and observations - the building blocks of science. The work is supported by modern internet tools such as open data bases, real-time collaborative editing software, wikis, and online computational models. These have been in active use in THL for years, and there is good expertise in such work.

In practice, each research question will have an own internet page on a collaborative web-workspace since the first day of the work. The answer to each question is iteratively built based on existing and new data, analyses, and discussions during the project. Anyone can participate in these discussions at any time, and the team members will moderate the discussions. The answers are updated regularly as new information arises, and the current best answer is available for users as open linked data at any given time. Web pages that are built in this way around relevant research questions are called knowledge crystals.

It is important to notice, that some of the research questions are designed in a way that they offer practical and direct guidance to relevant and timely policy issues. Knowledge crystal work should actively seek collaboration and contributions from policy makers to develop relevant questions and to include policy perspective to the work. Knowledge crystals are a practical solution to the collaboration need on science-policy interface. This work is supported by more traditional methods of communication and collaboration, such as reports, policy briefs, stakeholder workshops, and press releases.

Knowledge crystals helping decision support

Recently impact assessments have been made to support the preparation of laws. This is needed, and the know-how and resourcing should be developed further. Knowledge crystals help with this work. Their basic idea and structure enable them being used directly or at least easily in a new, similar assessment. This saves resources and increases the quality of the content.

The national sector and expert institutions have huge resources and expertise in their field. However, their working pace doesn't go well with the quick pace of preparation of laws, and thus it's hard to get concrete and comprehensive expert support when it's needed. Open variables will help with this too, because they are precisely built by experts for decision impact assessments.

So far these knowledge crystals have been rarely produced, even though the conditions are often good for it. Take for example Simuloitu Suomi aka Sisu-model. It is a comprehensive microeconomic description of how taxation and social support effect the population in Finland. However, it is a complex and huge model and there are only a handful of experts in Finland who can use in with a few days' notice. Most of the information from the model is thus only usable if one of these experts has the time to use the model exactly when the knowledge is needed. A special problem is the population data required for the model. It's not accessible by other people because of privacy protection issues.

As a solution to this example problem is that instead of using real population data, the model uses a statistically equivalent fake data that doesn't involve the privacy protection problems. Then the model could be built perfectly workable and open in the internet. It could also be built so that anyone could run the sub-models or look for precise information produced by model runs. When the amount of users increases, also know-how increases and expertise can spread wider and do more. This is possible if the model is updated to consist of open variables, aka knowledge crystals.

Research institutions could aim their operation in a way that one central product (along with expert reports and scientific publications) would be knowledge crystals.

The use of knowledge crystals creates the following action propositions:

• Action 5. Producing knowledge crystals is added to the goals of national research institutes. The progression of this operation model in different institutes is followed annually.
• Action 6. Education on producing knowledge crystals is held for experts in all sectors.
• Action 7. A law preparation project is chosen for which the preliminary assessment is done by using impact assessment based on knowledge crystals. Things learned from the trial is used in following projects.
• Action 8. Sisu-model (or some other significant model) is turned into a model working in the internet with knowledge crystals. Simultaneously experiences are collected about open use and development of a model large and complex but useful in decision-making.

Respect theory to motivate knowledge production

The respect theory is a draft of what kind of social reward system would encourage people and communities to work for a better society. Respect is measured in onors. The amount of onors given for a certain respectable act is figured out by asking: "How valuable is the act from the point of view of the society, meaning how many onors should the person doing it recieve? Note that onors don't measure financial gain, but is measured separately and measures the part that isn't measurable by money."

Benefit, responsibility or virtue?

The respect theory doesn't say what kind of ethics should be followed: whether the respectability of an act comes from the benefit (Joth Stuart Mill), the meaning (Immanuel Kant) or virtue (Aristoteles). What's relevant is that the act is respected in the society. It's also relevant that respect is measured orthogonally in respect to economic gain. Some act can be economically very profitable, but is indifferent in a moral or aesthetic sense. That's when the economical meter, money, itself tells about the meaning of the act to society, and respect is zero. For example creating jobs and finding meaningful things to do to people brings companies onors, but making profit doesn't.

How is respect measured in practise?

There are two parts to measuring respect. On one hand it is estimated how much respect should be given for a certain act, meaning how different acts relate to each other in the eyes of respect. On the other hand evidence that someone has done an honourable act is evaluated. To the theory the first question is more important. As to evidence, it's mostly just assumed it exists and is valid. Because my purpose is to first apply the respect theory to openly published data, the evidence won't be a problem: through server logs it's possible to trace who has published some knowledge for everyone to see and where and when this has happened.

The principle of measuring respectability is simple. People interested in issues of the society come together in the internet to discuss the respectability of different acts and the arguments for it. After that every one can give the amount of onors they feel is right for the act by answering the question above. One's respect for an act can be changed over time if needed. This is how the respectability of an act is constantly determined in the markets. So that the discussion could get started, the value of one onor will be set at the amount of acts helping society that a citizen can be assumed to do during one day. Or, adapting the guideline of scouts: do one good deed a day, get one onor.

How do onors relate to money?

The purpose of the respect theory is to motivate people to do acts that help the society, and the purpose of onors is to work as moral rewards on their own. The onors one has earned are public information, and the society's task is to keep up the respect given by onors. However, doing good acts takes people's time away from other things, like a paying job. It might be beneficial for the society to support economically people who want to concentrate on doing things that gain you onors, but in the current economical system doesn't gain you money. Here is a proposition to be a base for a conversation:

• A person earns onors as follows

$o_t = \sum_i o_{i,t} n_i,$

where o_t is the amount of onors a person earns at time t, o_i is the amount of onors a person earns by doing the act i, n is the number of good acts provably done by a person and t is a moment in time. It is worth noting that the onors for the same deed can change in time as the values of the society change.

• A person can reclaim their onors in money as follows:

$m_{t,i} = n_{i,k=2} a_t (\sum_i o_{i,t} \sum_{k=1}^2 n_{k,i})^{b_t}$

so that

$n_i = \sum_{k=1}^3 n_{k,i},$

where m is the amount of money the society pays for the onors reclaimed in the case, a is the "conversion rate" for onors (euros per onor^b), b (0 < b < 1) is a progression coefficient, and k = 1,2,3 so that 1 means the acts from which the onors are already reclaimed, 2 those that are now being reclaimed and 3 those that have not yet been reclaimed. A person can so reclaim the monetary value of the acts they want. Onors stay with people permanently, but after reclaiming them they no longer hold economical value. Unlike the total number of onors, the amount of reclaimed onors is not public information. In the equation the b in the power is meant to cause progression, because the reclaiming of money is supposed to support the long lasting work of big masses of people instead of rewarding with huge amounts of money those who are able to create some new, society changing innovation. The smaller b is, the steeper the progression and the more small amounts of onors are valued. If b = 0 what we're talking about is actually a civil salary, because the respect for ones acts doesn't affect the sum paid.

When calculating the progression those acts are taken into account that have been reclaimed before or are being reclaimed now, but not those ones from which onors have been received but not yet been reclaimed. This is to avoid a situation, where new onors lower the economical value of previously gained onors and thus would tempt to reclaim them for tactical reasons. However, it is important to note that the economical value of onors are calculated based the rates for acts at the moment of reclaiming. This is because the value of many acts are often not understood until a lot later, and so the current rate is societally thinking better than the rate at the moment of the act or some other moment in time. This is true to both the reclaiming rate and progression coefficient. This is why room for speculation is left and cannot be removed.

Why should society build a respect system?

Economists have said that middle-class work is disappearing. In the future the most secure jobs are either expert tasks, that produce especially a lot of value, or the kind of practical jobs like barber or nurse that can't be moved abroad. Digitalisation and globalisation threat many middle-class current professions. However, sensible and meaningful social work isn't disappearing from the world. Instead the resources of the current economical system to invest in this kind of work ends, because from the point of view of market forces opening data, reading to children and listening to your neighbour's worries doesn't create demand on a market that runs on money. The respect theory offers a solution that doesn't involve the reader, the child or the neighbour having to pay for others to help them. Instead the motor would be people's knowledge that the society finds these acts respectable and has the practical motivator of handing out onors for good deeds, which also help one's economical situation. This is how a balance is found: people are motivated to do good to each other, and the society encourages this by offering meaningful experiences and by giving resources to support the activities.

Onors motivating governance and sector institutes

Social knowledge work isn't easily bent to the current copyright system that is based on preventing the use of information unless the user pays the producer of the information a compensation. However, it is seen in research that the highest benefit for a society is reached when knowledge is as wide spread and as well available to everyone as possible. Thus it's interesting to try out a reward system that doesn't limit but encourages the sharing of information.

Onors can be used in producing and using scientific knowledge, if the principles and practises of giving out onors and the respecting of the people who get them are agreed on. Finland could act as a visionary in freeing scientific knowledge, if onors are given real value in performance conversations and in position appointments. Currently almost the only thing giving a researcher merit is a scientific peer-reviewed published article. Even though international practises change slowly, it's possible to move faster in Finland. Especially the government can make a difference to what things are emphasized in the ministry-governed expert and sector institutes. For example it would be justified that a researcher gains onors when they publish a research idea, the test setting, raw data, statistical analysis of someone else's data, or an article. Currently only the last is a formal merit.

Especially it would be possible to give onors to researchers and experts, who actively produce valid knowledge directly for decision support. On top of traditional expert work this includes producing knowledge crystals aka open variables or participating in building shared understanding.

The respect theory creates the following action proposals:

• Action 9. A task force is set up to create rules for measuring respect for the knowledge work in decision support for the government. The task force is in charge of the work, which is done openly online and aiming for a shared understanding.
• Action 10. The ministries bring opening knowledge and the merits that are given for producing it into conversation in the institutes they govern. The recommendations of the respect task force are taken into use in result guidance.
• Action 11. Research institutes take onors related to opening data into use in personnel performance reviews.
• Action 12. Onors are taken into use also in the preparation of laws, so that anyone taking part in the preparation can earn onors with their actions.
• Action 13. The government starts a campaign to bring into use the described motivation structure in Finland. The costs and other effects of taking the system into use is investigated.

Rationale

Here is conversation about what different actions can be taken. The best ones will be taken into the result.

• Built slowly during the entire work based on previous parts.
• Deliverable: Practical actions to carry out, so the use of knowledge in decision-making gets better.
  • Actions by VNK (incentives, developing the governing processess)
  • Actions by national research institutes (the forms of knowledge and its production, decision support, in for example the for of assessments)
• What should the implementation plan be like?
  • Bold enough to draw attention and create conversation. Not too utopistic, so it is still possible to turn into actions. It shouldn't be too obvious or mainstream, because then no added value is gained.
  • It should be integrated in some already existing programmes or processes, so the integration would occur smoothly. The first things that come to mind are the government's impact assessment development project and VKN's impact assessment group.

See also

Yhtakoytta-project

Project topics	Project main page · Task 1. The methods and tools for evidence-based decision making · Task 2. Development needs for evidence-based decision preparation · Task 3. Agile experiments of evidence-based decision making · Task 4. Implementation plan of evidence-based policy · Task 5. The communication and working plan for Yhtakoytta-project · Task 6. The final report of Yhtakoytta-project
Methods and tools	Open policy practice · Open assessment · Open policy practice municipal pilot (in Finnish) · Assessent of environment-caused health problems (in Finnish) · Tekaisu-project · Category:Tekaisu
Publications and other things useful	Yhtäköyttä (Project introduction in Finnish) · op_fi:Avoin päätöksentekokäytäntö voisi parantaa tiedon hyödyntämistä (in Finnish) · op_fi:Ympäristö- ja terveysvaikutuksia koskeva tieto kunnallisessa päätöksenteossa (in Finnish) · All pages in category Yhtakoytta
This page is produced as a part of the Finnish Government's research plan assignment in 2015 (www.vn.fi/teas). The producers of the information are responsible for the contents. The contents do not represent the views of the Government.