Forecasting for Policy: helping public institutions to understand the future
Information about judgemental forecasting and our forecasting project in the Czech Republic
"Forecasting is an interesting approach to aggregating views on possible future developments that is useful in personal and public decision-making."
FORPOL is a joint project between Czech Priorities and Metaculus. It is funded by the Effective Altruism Infrastructure Fund. It aims to increase our understanding of practical ways, how to improve public policy-making with the use of judgemental forecasting.
The main objective of the FORPOL (Forecasting for Policy) project is to provide public institutions with useful predictions on relevant societal topics. We currently operate forecasting tools in the Czech republic, but are interested in establishing international cooperations as well. FORPOL is also an opportunity for (currently only czech speaking) participants to improve their forecasting skills.
We are using judgmental forecasting, a method for obtaining estimates of future scenarios based on probabilistic reasoning and aggregation of a large number of qualified forecasters. Judgmental forecasting is especially appropriate for predictions on novel issues or where there is insufficient data (e.g., societal impacts of a pandemic, new technologies, etc.). It is also used on highly complex topics such as; geopolitics, crisis response and other social challenges.
Judgemental forecasting is based on quantitative data but also takes into account qualitative analysis. It thus presents the possibility of assessing the situation on the basis of currently available information.
A key element of the judgmental forecasting method is collective intelligence - instead of relying on the judgment of just one expert it estimates the outcome as the result of a group of forecasters aggregated predictions. This way of forecasting minimizes information noise and bias and, at the same time, maximizes the volume of information on which the aggregated forecasts are based.
Benefits of forecasting tournaments
Compared to other methods of aggregating expert opinions (interviews and panels, questionnaires, Delphi), forecasting tournaments are focused on three aspects:
- Accuracy: By using numerical or probabilistic predictions instead of vague verbal expressions ("most likely", "very likely", etc.), it is possible to share more accurate information and subsequently evaluate and modify one's own predictions. These probabilities also make it possible to assess the accuracy of predictions and, after a period of time, identify consistently above-average forecasters whose predictions can get more weight in the model overall score based on objective criteria.
- Balance: By using an online platform, it is possible to involve more participants with diverse expertise and experience to boost information sharing.
- Engagement: The online platform allows forecasters to further comment about the questions they are solving. Participants have strong incentives to get involved in these discussions through rewards which elevate their prediction score.
The online platform automatically collects all the inputs, which serve as a digital repository of views and resources to improve prediction skills within the forecaster's community.
What did we learn in FORPOL?
The aim of the FORPOL (Forecasting for policy) tournament was to provide public institutions with useful predictions on important societal topics.
The FORPOL tournament ended on 5 April 2023.