I want to start making some custom alert scripts. As usual, I like to practice by using a live example. I have SSH remote access and Apache enabled on my laptop. When at work I keep a map up in Splunk on my laptop showing the source ip location of any attempts to connect to my laptop. If you start beating on my laptop it results in an instant ban hammer in the network IPS.
I sometimes miss seeing the map updates when busy. If I had an alert history that is quickly accessible it would be easier to handle the scanning systems. I decided on this alert to test the hits on apache that runs every 15 minutes. These logs just happen to go into an index called os_osx. I tagged the combined_access source type as “web”.
index=os_osx tag=web | stats count by clientip
Now the fun part. I am working on my python skills so I did the alert script in python. This required me to call the OSX shell command osascript in order to execute the Apple Script that generates the actual Notification Center message. It took a minute of experimentation to get the right combination of escaped quotes to build the Apple Script command.
We get a result like this:
And here is the alert script that I saved as osx-alert.py in the /Applications/splunk/bin/scripts folder on my laptop. That is the script I chose to call on the search above when saved as an alert.
from subprocess import call
if __name__ == "__main__":
# Obtain the path to the alert events compressed file
alertEventsFile = os.environ['SPLUNK_ARG_8']
# Handle to the csv contents of the alerts events compressed file
eventContents = csv.reader(gzip.open(alertEventsFile, 'rb'))
# Assign the contents to a list iterator and skip the header line of the table.
alert_iterator = iter(eventContents)
# Send a notification for each source ip in the alert results table. We grab the IP and count from the columns in each row of the stats count csv format output from Splunk.
for line in alert_iterator:
message = "ALERT: "+line+" connections from ip: "+line+" in past 15 minutes."
call(["osascript","-e","display notification \""+message+"\" with title \"Splunk\""])