ThreatPlaybook - Guide

Installation Instructions and QuickStart for ThreatPlaybook

This is still a pre-release version and might be buggy :) However, we encourage you to use it. Please let us know if you need any help with this

Installation

Components Required

• MongoDB 3.X

• Python 2.7 (Sorry this is still not 3.0 because several important libs in Robot Framework don't support Py3)

• Robot Framework Libs as required

• Mermaid CLI (for Diagrams)

  • Mermaid CLI is a Node application and needs to be installed with the instructions given here: https://github.com/mermaidjs/mermaid.cli

  • I'd highly recommend that you install Mermaid CLI globally with: npm install -g mermaid.cli

Installation

• Create virtualenv in the directory you want to run ThreatPlaybook in

• Make sure that MongoDB is installed and running

• Install ThreatPlaybook by running pip install ThreatPlaybook

• Once ThreatPlaybook is installed, you need to run a command to create some boilerplate directories and files for your project. These would be the directories and files you need to capture Use Cases, Abuse Cases, Threat Models and so on

• To create the boilerplate, run the following:

  • threat-playbook <ProjectName>

  • This creates the following directories in the current working directory with some boilerplate yaml files and a single robot Test Suite file

Your Folder Structure must look like this:

- cases (dir)
- security_tests (dir)
---- security_tests.yml
- entities (dir)
---- entities_connections.yml
- results (dir)
- <ProjectName>.robot (file)

Now you are ready to start. Let's understand some of the key files and components you need for this

Key Components

.robot Test Suite

This is the Test Suite file that will run all the test cases within the file and produce the desired outputs. Nearly all the outputs are stored in the MongoDB instance that you have setup.

This is the file that robot framework will use to run the:

• processing of various threat models, security test cases, abuse cases and user stories

• run automation of different security tools (with Robot Libraries) like Zap, Burp, Nmap, etc

Basically, this is the file that controls the entire flow of ThreatPlaybook for your project

Resources

• Please see the example TestRobot.robot for additional references.

• Consider learning Robot Framework with a quick Robot Framework Tutorial

entities_connections.yml

This file captures all the entities and their connections. Entities are any elements or objects that you would use to capture process flows in your Threat Model. Entities are typically functionality/pages/features within the application. They are connected to other features and functions and data flows and there's data shared between these entities

For example, below is a simple process flow in an application that is used for Expense Management Application

User logs in (Login Feature) => Dashboard(Feature) => Add Expenses (Feature) => Upload Expense File(Feature)

Details of all the entities and connections of the entities are captured in this yaml file.

entities:
  user: #this value needs to be unique, and without whitespace characters
    description: Public End-User accesses over the Browser
    caption: User with Browser #this is what gets printed in the process flow diagram
    name: user
  login:
    description: End-User authenticates to Front-end with username and password
    caption: Login with Username/Password or Token
    name: login
  manage_expenses:
    description: User can upload and view expenses uploaded by him/her
    caption: View/Create/Upload Expense Information
    name: manage_expenses

The second section of this file captures all the connections between the different elements/entities. These are captured as list values under the key connections

The following are the elements in the list

• list index0 => start entity

• list index1 => end entity

• list index2 => data exchanged between entities

• list index3 (optional) => Trust Boundary (This hasn't been incorporated into this release, but is planned for the future. Please see our TODO list)

connections:
  - [user,login, username&password/token, Authentication credentials]
  - [login, manage_expenses, expense info, Cut the Funds App]
  - [manage_expenses, s3, expense files, Amazon S3]
  - [login, profile, User PII/Credit card info, Cut the Funds App]
  - [login, logout, Logout, Cut the Funds App]

When the keyword write markdown report is invoked in the robot test file, it generates a nifty process flow diagram like this

Generated process flow diagrams when the Test Suite is finally done running

Please see example entities file: https://github.com/we45/ThreatPlaybook/blob/master/example/entities/entities_connections.yml

We are currently not using Entities and Connections for anything else other than generating the process flow diagram, but the plan is to use it to link it back to Threat Models in a later release. Please look at our TODO list

security_tests.yml

This is a file that can you capture all the different security test cases. The idea is that this file can be easily added to or deleted from. Internally, we use a security_tests file to create multiple security tests per project. The good thing with the YAML file is that you can easily copy-paste many of the generic test cases into different projects

Security test cases are not mandatory but are very useful for Pentesters to track the coverage of their pentests against a pre-defined set of test cases

Each Security Test has a name, description, a type and tags.

• The name must be unique

• The type can be either: (A)utomated/(M)anual/(R)econ (reconnaissance). We may choose to do away with this moving forward

• Tags can be terms that you want to tag the security test case by. We have implemented a minimal relationship, where you can tag a recon test case to a recon test (actually running an nmap or ZAP) based on the tag. This needs improvement

sql_injection_auto: #this needs to be a unique value as well
  description: Use Automated Vulnerability Scanners to test for SQL Injection
  type: A
  tags: zap,burp,arachni

template_injection_auto:
  description: Check for Template Injection with Burp
  type: A
  tags: burp,tplmap

sql_injection_manual:
  description: Manual SQL Injection Tests
  type: M
  tags: zap,burp,arachni

sql_injection_sqlmap:
  description: Attempt to Exploit SQL Injection with SQLMap
  type: A
  tags: manual

Please see example Security Test Cases file: https://github.com/we45/ThreatPlaybook/blob/master/example/security_tests/security_tests.yml

cases (Directory)

This is the most important directory, as far as Threat Models are concerned. What you should do is to create separate YAML files for each of the User Stories/Features that you want to capture. When the test suite runs, it looks for all *.yml files in this directory and processes them as Use-Cases, Abuse Cases and Threat Models.

Please refer to the comments succeeding the #value to understand the structure of this yaml file. This is a single user story yaml file.

login_user: #this is the short_name for the User Story/Functionality
  description: |
    As an employee of the organization,
    I would like to login to the Expense Management application to submit and upload expense information
  abuse_cases: # The Key for all abuse cases under the User Story/Functionality
    external_attacker_account_takeover: #unique name for Abuser Story
      description: As an external attacker, I would compromise a single/multiple user accounts to gain access to sensitive corporate information, like expenses
      threat_scenarios: # Key for all Threat Models under Abuser Story
        sql injection user account access: #Unique Threat Model Name
          description: External Attacker may be able to gain access to user accounts by successfully performing SQL Injection Attacks against some of the unauthenticated pages in the application
          severity: 3 #value from 0-3 0 is lowest on severity, and 3 is High Severity
          cwe: 89,90 #required CWEID if you want to correlate and map to vulnerabilities identified
          cases: #linked test cases (optional)
            - sql_injection_auto #the same name as the ones in security_tests
            - sql_injection_manual
            - sql_injection_sqlmap
            - generic_error_messages
        end user weak password:
          description: External attacker may be able to bypass user authentication by compromising weak passwords of users
          severity: 2
          cwe: 521
          cases:
            - default_passwords
            - bruteforce_login
        end user default password:
          description: External attacker may be able to bypass user authentication by compromising default passwords of users
          severity: 3
          cwe: 284
          cases:
            - default_passwords
        auth token hijacking mitm:
          description: Attacker attempts to compromise auth token by gaining access to the end user's auth token by performing Man in the Middle Attacks
          severity: 2
          cwe: 295
          cases:
            - ssl_checks

Few points to Consider

• CWE-IDs are important. For those who aren't aware. Most Application Vulnerabilities are associated with a Common Weakness Enumeration ID value, which details the vulnerability. Example SQL Injections are CWE: 89. CWEs are required for ThreatPlaybook to correlate Vulnerabilities to Threat Models. For example - If you run ZAP and it identifies SQL Injection, you can correlate it with the Threat Models by CWEID. Else, there's no correlation possible. More details here: https://cwe.mitre.org/

Please refer to the example use-cases yaml files in this directory: https://github.com/we45/ThreatPlaybook/tree/master/example/cases

results (Directory)

This is where all the results and reports are written to. In the Test Suite file, the results directory has to be explicitly specified for tools to write their results into.

Demo of ThreatPlaybook against Demo App