Limitations of Safety Assurance and Goal Structuring Notation (GSN)
"People are putting a lot of effort into figuring out how to assure a system is safe after the system design is completed. This white paper presents some of the difficulties and alternatives to emphasizing after-the-fact assurance of safety"
Quelle
| Format | Veröffentlichung | Link |
|---|---|---|
| Whitepaper | 2020 | http://sunnyday.mit.edu/safety-assurance.pdf |
Wahrgenommen von Heinrichsgeist: 2023. Zitate auf dieser Seite beziehen sich auf diese Quelle, sofern nicht anders gekennzeichnet (Zitationszweck: Anschauliche Hervorhebung ausgewählter Passagen).
Interessant
- Safety kann nicht im Nachhinein verargumentiert werden
- GSN ist Anfälligkeit für confirmation bias
How can we overcome confirmation bias? One way is to look for ways to challenge your goal, not to achieve it. For example, look for arguments that the system is not safe rather than arguments that it is safe. Making an argument for safety is always subject to confirmation bias. Instead you need to focus on gathering evidence that the system is unsafe. Actively seek out and consider contradictory evidence. This is the standard process used in hazard analysis. Note that structuring the confirmatory arguments in some notation made up of boxes and arrows or arcane mathematical symbols does not make them less problematic; it simply provides false confidence that they are correct and unbiased. In some sense, structured arguments may be more dangerous.
- GSN soll etwas beweisen, Verweis auf englische Wikipedia
- Argument von Listen/Textform in grafische Notation zu bringen
- Beispiele für poor GSN Arguments mit logischen Fehlern
- conclusion
At best, GSN and other formatted notations for arguments add nothing except the cost of putting the argument in this form. At worst, they are misleading and dangerous and subject to confirmation bias. And simply documenting standard processes in a tree structure rather than textually provides no added value.