Proper modeling of human behavior is crucial when developing agent-based models to investigate the effects of policies, such as the potential consequences of interventions during a pandemic. It is, however, unclear, how sophisticated behavior models need to be for being considered suitable to support policy making. The goal of this paper is to identify recommendations on how human behavior should be modeled in Agent-Based Social Simulation (ABSS) as well as to investigate to what extent these recommendations are actually followed by models explicitly developed for policy making. By analyzing the literature, we identify seven relevant aspects of human behavior for consideration in ABSS. Based on these aspects, we review how human behavior is modeled in ABSS of COVID-19 interventions, in order to investigate the capabilities and limitations of these models to provide policy advice. We focus on models that were published within six months of the start of the pandemic as this is when policy makers needed the support provided by ABSS the most. It was found that most models did not include the majority of the identified relevant aspects, in particular norm compliance, agent deliberation, and interventions’ affective effects on individuals. We argue that ABSS models need a higher level of descriptiveness than what is present in most of the studied early COVID-19 models to support policymaker decisions.