Security/Projects/Minion

Minion is a security testing framework built by Mozilla to bridge the gap between developers and security testers. To do so, it enables developers to scan their projects using a friendly interface.

NOTE - this project is at a very early stage in its development.

Source code: https://github.com/ygjb/minion - public
Task management: https://trello.com/b/DlVPzGaS - currently private, contact one of the Minion developers to get access

Developers:

Psiinon
TBA

Initial Diagram

Components

Web Interface

Overview

The Web UI is responsible for:

Generating the web UI (not surprisingly)
Authenticating and managing users and user sessions

Notes

Log in using Persona (BrowserID) (can be restricted by domain for use on central server by organizations)
Menu -> New Scan, Running Scans, Completed Scans
- Future: Group Scans (member of groups, permissions, see other scans by group members/project)
New Scan
- Basic: URL, Port
- Advanced: Login information, technologies used (customize scan such as SQLmap for SQL)
- Future: Scan type based on plugin (web app, client code, etc)

It should maintain as little data in memory as possible - all data should be retrieved from the Task Engine (and/or db?). This will allow us to run multiple Web UI servers for one service.

Questions

Should this also provide a REST based API, or will we rely on the one implemented by the Task Engine?
Will it need access to the db or will it get all data from the Task Engine?

Task Engine

Overview

The task engine is responsible for:

Managing Minions
Persisting all info to the db
Providing a REST API

Notes

Instance started when user clicks start scan
Collects provided information
Starts scan based on provided information
Launches tools (Minions) and awaits responses

It should persist all data to a db and maintain as little data in memory as possible. This will allow us to run multiple Task Engine servers for one service, with all of the synchronization happening via the db (which would probably be clustered).

Minions (Scanners)

Receive kickoff from task engine
Scan target
Send results back to task engine in necessary format (JSON)

Target

Site hosted by developer
Can be hosted or running locally

Configuration Abstraction

To pass data between components, a standard configuration and data scheme must be used. For Minion, we have decided to use JSON and a REST API as the format for passing data. For example, the user interface will collect the necessary options, then call the task engine and pass it a JSON string of those options. The task engine will read the options and use each installed tool's REST API to make requests to the tool (some tools may need different options than others). The tool will pass its results back to the task engine as a JSON string and the task engine will compile all returned results into a single JSON string which will be returned to the interface. The interface will deconstruct the string into human-readable results and display them on the results page.

Basic Tool Requirements

Each tool takes a number of base options as well as optional options that can improve the scan's accuracy or increase its functionality.

Zed Attack Proxy

Base options: URL

Optional: spider depth-level, CSRF tokens, authentication information, parameters to fuzz

Garmr

Base options: URL

Optional: parameters to test

Note: most options with Garmr involve output, which should be handled without interaction from the user.

Skipfish

Base options: URL

Optional: authentication credentials, cookie values, non-standard header information, scan time limit (see http://code.google.com/p/skipfish/wiki/SkipfishDoc for a full list)

Note: Skipfish has a lot of additional options that include domains to exclude in crawling, domains to ignore in testing, wordlist generation, folder output, etc. To make it as easy to use as possible, the Skipfish plugin should include defaults for all of these options so that the user will never need to worry about the options.

Installation Notes

PYTHONPATH="$PYTHONPATH:$HOME/minion/task_engine" PYTHONPATH="$PYTHONPATH:$HOME/minion/plugins" export PYTHONPATH

sudo easy_install bottle