Malmö University Publications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
[In]Credible Models – Verification, Validation & Accreditation of Agent-Based Models to Support Policy-Making
Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT). Malmö University, Internet of Things and People (IOTAP).ORCID iD: 0009-0004-1712-5181
Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT). Malmö University, Internet of Things and People (IOTAP).ORCID iD: 0000-0002-2399-8817
Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT). Malmö University, Internet of Things and People (IOTAP).ORCID iD: 0000-0002-8209-0921
Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT). Malmö University, Internet of Things and People (IOTAP).ORCID iD: 0000-0003-0998-6585
2024 (English)In: JASSS: Journal of Artificial Societies and Social Simulation, E-ISSN 1460-7425, Vol. 27, no 4, article id 4Article in journal (Refereed) Published
Abstract [en]

This paper explores the topic of model credibility of Agent-based Models and how they should be evaluated prior to application in policy-making. Specifically, this involves analyzing bordering literature from different fields to: (1) establish a definition of model credibility -- a measure of confidence in the model's inferential capability -- and to (2) assess how model credibility can be strengthened through Verification, Validation, and Accreditation (VV&A) prior to application, as well as through post-application evaluation. Several studies have highlighted severe shortcomings in how V&V of Agent-based Models is performed and documented, and few public administrations have an established process for model accreditation. To address the first issue, we examine the literature on model V&V and, based on this review, introduce and outline the usage of a V&V plan. To address the second issue, we take inspiration from a practical use case of model accreditation applied by a government institution to propose a framework for the accreditation of ABMs for policy-making. The paper concludes with a discussion of the risks associated with improper assessments of model credibility. 

Place, publisher, year, edition, pages
European Social Simulation Association , 2024. Vol. 27, no 4, article id 4
Keywords [en]
Policy-Modelling, Model Credibility, Accreditation, VV&A, Agent-Based Modelling & Simulation, ABM4Policy
National Category
Computer and Information Sciences
Identifiers
URN: urn:nbn:se:mau:diva-71919DOI: 10.18564/jasss.5505ISI: 001349760200002Scopus ID: 2-s2.0-85209081992OAI: oai:DiVA.org:mau-71919DiVA, id: diva2:1910692
Available from: 2024-11-05 Created: 2024-11-05 Last updated: 2024-12-17Bibliographically approved
In thesis
1. Agent-based Social Simulation & Policy-Modelling: Facilitating Realistic and Credible Decision-making Support
Open this publication in new window or tab >>Agent-based Social Simulation & Policy-Modelling: Facilitating Realistic and Credible Decision-making Support
2025 (English)Licentiate thesis, comprehensive summary (Other academic)
Abstract [en]

This dissertation explores the use of Agent-based Social Simulation (ABSS) and policymodelling to support policy-making. ABSS, consisting of autonomous agents mimicking human behavior, offers a valuable alternative to traditional policy analysis methods by employing agent technology in the exploration of complex social systems and emerging behaviors. This approach allows policy-makers to perform simulated policy experiments in a safe digital environment, assessing potential adverse effects before implementation. Despite its potential, ABSS adoption in policy-making is limited. The main purpose of the dissertation is to better understand why this is the case and how the current challenges can be addressed to increase ABSS usage in policy-making.

Using a systematic review approach, six challenges in applied policy-modelling were identified: scope, politics, management, understandability, credibility, and data. It shows that upstream modelling decisions affect the rigor of model testing and highlights transparency issues like those prevalent during the COVID-19 pandemic. Furthermore, the dissertation highlights that the lack of formal accreditation and communication of model results poses a significant risk for faulty applications, which can cause societal harm. Addressing this weakness in the application-chain to increase the robustness of evidence-based policymaking is of the utmost importance.

Using a design science methodology, two artifacts were developed to address these challenges. The first includes a verification and validation protocol and an accreditation framework, facilitating independent credibility assessment of ABSS models. This design aligns model application with the scientific principle of independent review and strengthens the application-chain through quality assurance prior to application. The second artifact is a high-fidelity policy-modelling methodology employing the Institutional Grammar 2.0, ensuring systematic and transparent modelling using the case of organ donation. This methodology formalizes implemented regulations before involving subject matter experts, ensuring methodological stringency for the development of policy models. These artifacts seek to contribute to the development of realistic policy models and their responsible deployment as decision support tools within the public sector.

Place, publisher, year, edition, pages
Malmö University Press, 2025. p. 67
Series
Studies in Computer Science ; 31
National Category
Computer Sciences
Identifiers
urn:nbn:se:mau:diva-72795 (URN)10.24834/isbn.9789178775699 (DOI)978-91-7877-568-2 (ISBN)978-91-7877-569-9 (ISBN)
Presentation
2025-01-15, Lecture hall B2, Niagara,, Nordenskiöldsgatan 1, Malmö, 13:00 (English)
Opponent
Supervisors
Note

The papers are not included in the fulltext online.

Paper IV in dissertation as manuscript.

Available from: 2024-12-17 Created: 2024-12-17 Last updated: 2024-12-20Bibliographically approved

Open Access in DiVA

fulltext(1195 kB)81 downloads
File information
File name FULLTEXT01.pdfFile size 1195 kBChecksum SHA-512
9fed916e9944d39034e3d0447d65ec44f5f590fe1244b1b8e0bfe99974b2bb0a1af163c73e7530daedc85f3dfd8ae273e2309bb81d6ad987b862164f77884010
Type fulltextMimetype application/pdf

Other links

Publisher's full textScopus

Authority records

Belfrage, MichaelJohansson, EmilLorig, FabianDavidsson, Paul

Search in DiVA

By author/editor
Belfrage, MichaelJohansson, EmilLorig, FabianDavidsson, Paul
By organisation
Department of Computer Science and Media Technology (DVMT)Internet of Things and People (IOTAP)
In the same journal
JASSS: Journal of Artificial Societies and Social Simulation
Computer and Information Sciences

Search outside of DiVA

GoogleGoogle Scholar
Total: 81 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 244 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf