While mitigating an outage you may run into recurring problems that you don't have the engineering resources to address the underlying issues or just can't automate the remediation for a variety of reasons. Engineers commonly write one-off scripts to patch the problem that end up in GitHub Gists, local text files, Slack code snippets or even Google docs and they're frequently forgotten and recreated every time.
FireHydrant lets you store these scripts and track their execution, success and usefulness over time with our Runbook Execute a Script step. We'll show the raw script letting your engineers copy/paste it out into their terminal or give you a curl command to execute it and report the status back into FireHydrant. This lets you keep track of when the scripts are executed, by who and their output.
Add a step to your runbook with a description and then any Bash code you'd like your engineers to execute in order to mitigate an incident.
Your engineers will be presented with the script itself along with a
curl command allowing FireHydrant to capture the error status and output. This lets you record when the step was executed, by whom and if it was successful.
The script is available from Slack and our web UI.
If you run the script using our "auto execute" functionality, the step will automatically transition from pending to complete or errored when it's run by one of your engineers. You can see below that the error status (missing API token) was recorded by FireHydrant: