The Inaugural Community-Driven State of Threat Modeling Report

We are excited to announce that the State of Threat Modelling Report 2024-2025 has been (finally!) released. We hope you will find it a useful resource to learn more about how the threat modelling community practices the threat modelling activity, informing not just what is common and what works, but sharing the challenges we all face.  

How It All Started

It is that benefit to the community that has always been the underlying goal of the SOTM Survey, and that’s fitting as it was the community that served as the catalyst for the idea to have a survey. It was shortly after Dave joined the Threat Modelling Connect community in 2023 that he noticed it was possible to use the forum tools to conduct a poll, and so the question was asked “To DFD or not to DFD?” (sadly the results of the poll did not survive the change in forum backend), and that was really the spark for the idea “wouldn’t it be great to have a community run survey for threat modelling?”. Of course the idea of a survey is not novel, many industries leverage them, but what was required to make the survey a success was a community to reach out to in order to answer the survey questions, and so the idea remained just that, until the Threat Modelling Connect community was thriving and ready to embrace the task.

Designing the Survey

The work eventually began to start creating a set of questions for the survey, and it quickly became clear that creating surveys is much harder than it seems!  What topics should we cover? What kind of answers do we want?  Who will the target audience be? Will the questions make sense to people?  Are the answers complete and representative? Have we created questions and answers containing our own inherent biases? In the end the actual topics to cover were relatively easy to decide and a substantial set of questions were amassed relatively quickly as we drew upon our own experiences and knowledge of threat modelling to structure the survey to align with how the threat modelling activity is performed in practice. A guiding goal during this phase was that the focus be on gathering data for how companies threat model and not how individuals think threat modelling should be done. We very much wanted it to be the State of Threat Modelling practice and not the State of Threat Modelling theory. But of course in practice every company threat models slightly differently, and the activity flexes to the business it serves, which meant many questions would not have a finite set of answers (for instance it was assumed that few companies would follow a well-known threat modelling methodology rigidly) and so many answers tried to accommodate that expected variety of practice. Needless to say there was debate and compromise right up until the date we set ourselves to launch the survey.

Launching and Gathering Voices

And launch it we did, as far and as wide as we knew how, which turns out was probably not as far and as wide as it needed to go to reach every threat modeller on the planet, but we begged, cajoled and imposed on anyone with a platform that we thought would reach our target audience. Whether they are active members of the threat modelling community or not, we wanted to hear every threat modeller's voice. Ultimately we didn’t hit the highest of our expectations for the number of voices we wanted to hear, but for the first attempt at such a survey, getting threat modelling data from 73 responses relating to 60 companies wasn’t a bad result, and we think the only direction to go from here is up. If you have any ideas about how to spread the survey further and wider in the future, then please take the time to share your ideas.

Turning Data into Insights

Now, if you thought creating the survey sounded like a difficult task, then you (as we were) would not have been prepared for the completely different challenge of trying to process the results, analyse the data, and create a report that delivered on the goal of informing the community about itself. But having come that far, we were too stubborn to fall at the final hurdle and managed to cobble together a report which is available now for the community to digest. We think it is an interesting report; even as some of the data did nothing but confirm our own (and likely others') understanding of what is common in the community, because, and for the first time, it wasn’t a matter of speculation, but of information, serving as the foundation of knowledge. Do not fear though, as there will undoubtedly be at least one conclusion in the report that doesn’t align with your current beliefs, and you’ll be incredulous that some people don’t see things the way you do. But that’s good! - you can’t fix the problem until you understand it, and this report is a great place to start.

By the Community, For the Community

So go read the report. Digest the report. Talk about the report. Post about the report.

If the report doesn’t do enough to scratch your itch for information, don’t worry, we have made available the raw data collected as well. This means the community is free to perform its own analysis of the data and draw its own conclusions, perhaps even finding correlations we have missed (we are hoping you do!).

We hope this data and this report serve the threat modelling community. We hope it is the start of a regular event in the threat modelling community calendar that allows the community to look at itself today so it better knows how to achieve its goals in the future.