I would say this is the most enigmatic of the 4 questions in the 4 question threat modelling framework.
After you’ve clearly defined what you are going to threat model, after you’ve analyzed the system for threats, after you’ve created a set of additional controls to mitigate those threats - did you do a good job? How do you know? What does “good” even mean? Would you really ever finish a threat model and then call it ugly?
It’d be fascinating to hear how people tackle this thorny question?
I view that question more in the agile sense-- "what could we do better?" "How can we improve?" That turns it into more of an iterative process without a final answer.
Remember the TM exercise is usually done as a group (“Did *WE*”), so the dynamic is prone to inconsistencies you should check
When I teach TM I include the following checks:
In addition you should update the original system diagram:
And your own global TM assumptions and catalog of Treats, Threat Actors, Risks, Controls, Checks
At the end, the model (subjective):
In “A Guide to Threat Modelling for Developers” (https://martinfowler.com/articles/agile-threat-modelling.html) Jim Gumbley said:
“Adam Shostack, who has written extensively on threat modelling and has provided feedback on this guide takes credit for the three question structure. He also adds a fourth question "Did we do a good enough job?" I don't disagree with Adam that we need to reflect and improve on the outcomes of our threat modelling. However, I have omitted this question from the basic structure as I believe it can be addressed elsewhere. Iterating and improving based on feedback should be implicit in agile software development, particularly when we are threat modelling 'little and often'.”
For me this question is the most enigmatic and the most difficult to answer.
As a rule of thumb, the first thing we need to be clear about is the type of system we need to protect. Depending on the level of criticality, we should adjust the time, efforts, and resources so the task is not endless.
From here, my advice is to try to see things from different points of view. Different profiles can provide a different perspective that can help identify aspects that had not been taken into account.
I see the “Did we do a good job” more as how did we do. To understand what good looks like, would itself need some thought or planning and invariably, over the course of execution, things change right!? Agile approach therefore as mentioned would be a suitable approach and depending on the technology and processes being used greatly impacts the outcome. Considering these, if improvements have been identified, and there is scope for further improvements (next iteration), then we did a good job.
Not a member yet? Become a member to join forum discussions, participate in community events and apply to write articles.
Create an accountEnter your username or e-mail address. We'll send you an e-mail with instructions to reset your password.
