Beverly and McLoughlin (2019) compared observed burned areas from 138 fires in Alberta to simulated burn probability heat maps generated prior to the fires.  Results showed that observed burned areas did not occur preferentially in burn probability hotspots mapped with the Burn-P3 model. The study also reported that performance of BP maps depends on subjective decisions about how you displayed the raw Burn-P3 values (i.e., the classification scheme you chose). A quantile classification resulted in the best performance, especially when bin number was limited to 3 classes, but this produced vast BP regions with little insight for prioritizing localized mitigation measures.

The methods used by Beverly and McLoughlin (2019) followed a previous study (Paz et al. 2011) that compared one fire in Israel to a pre-fire burn probability map and reported good agreement. In a journal article published in 2013, supporters of the Burn-P3 model (Marc Parisien and Xianli Wang) recommended and endorsed the approach used by Paz et al. (2011) for assessing the accuracy of Burn-P3 maps. In Parisien et al. (2013), they noted the encouraging results of the Paz et al. (2011) study. In contrast, the Alberta study by Beverly and McLoughlin (2019) ​showed poor alignment between real-world fires and pre-fire Burn-P3 simulations.

An important conclusion of the Beverly and McLoughlin (2019) study was that Burn-P3 performed poorly when model outputs were mapped with the spatial precision that managers would need to inform fine-scale, local management actions such as prioritizing the locations of fire control lines or evacuation routes. Given the potential implications of these results for fire management decisions that can have profound public safety repercussions, Beverly and McLoughlin (2019) call for caution and further evaluation of the Burn-P3 model.

In response to the apparent shortcomings and limitations of burn probability simulations,  Beverly, McLoughlin, and Chapman (2021), have developed a new landscape fire metric that aligns well with real-world fires. Read more about it on our Landscape fire exposure page.

Recently, researchers in Alaska provided more evidence that simulated burn probabilities do not align with real wildfires. In Schmidt et al. (2024), they compared real wildfires to simulated burn probabilities from FLAMAP (refer to Table 4).