Splunk and Apple Swift for Alert Scripting

This week, I attended my first Nashville Cocoaheads meeting in a few years. Cocoaheads is basically an Apple programming club. The topic was “Scripting with Swift” It was a great presentation by Blake Merryman. Blake works for Griffin Technology and they were our awesome hosts for the meeting. Griffin is one of my favorite accessory makers. Plus there are right here in Nashville.

Of course I immediately wondered. If I can treat Apple Swift as a shell scripting language… Splunk alerts written in Apple Swift! Oh yeah why not?!?

And… The short answer is that it works for Splunk running on OSX. This means you could make data sent from Splunk via alerts to OSX code that has full native library access to anything on the Mac.

Here is a very simple example thanks again to Blake’s example code.

Alert Script Setup

You must have Xcode installed on the Mac as well as Splunk. Xcode provides the Swift language installation.

You need to make a shell wrapper script in the $SPLUNK_HOME/bin/scripts/ folder. Splunk won’t be able to call a .swift script directly. You will need to add the executable flag to both script files. Then you simply setup the desired Splunk alert to call alertSwift.sh which is just a shell command wrapper script to call the Swift script. The key on the swift script is the hashbang that tells the system how to execute Swift.

Our Swift code example below is a simple call of Apple Script to make the Mac say “Splunk Alert.” This is a very simple example. But it shows that if you use all the normal tricks of pulling in the alert script arguments you could pull in a search results zip file and take actions in Mac OSX based on the data. It could be anything including Notification Center alerts. Enjoy the possibilities.

Alert Script Code

Example alertSwift.sh: (Make sure you do a chmod +x alertSwift.sh)

Example alertSwift.swift: (Make sure you do a chmod +x alertSwift.swift)

Splunk .conf 2014 Slides

** Update Oct 15, 2014 – Poodle SSLv3 Issue**
The talk was given the week before the SSLv3 issue was released. Please remove all references to supportSSLV3Only = true from the configs when you use them. You also can find more from Splunk on the SSLv3 issue and how to mitigate at http://www.splunk.com/view/SP-CAAANKE


.conf 2014 was a great time this year. Duane and I enjoyed giving the talk “Avoid the SSLippery Slope of Default SSL” with great questions from the audience. I was surprised at the solid turn out for a Thursday 9am talk. My talk was “From Tool to Team Member: Controlling Systems with Splunk Alert Scripts”

Here are the PDF copies of the slides for both talks:

Increasingly, production security requires more than using default SSL certificates. This session will cover best practices for implementing your own SSL certificates on all Splunk channels. The right configuration and steps can provide both encryption and authentication needed for today’s due diligence requirements.

We will go in depth into setting up alert scripts that can make web services calls to other devices such as intrusion prevention systems. This gives Splunk the ability to actively control such systems. Code samples will be provided that include being able to save login credentials encrypted within Splunk. Using alert scripts we can change Splunk from just a tool into an IT team member taking actions on your behalf!

Splunk a DNS Lookup for Abuse Contacts

Let’s follow up on our DNS theme of the last post. I have used my alert scripting to block attackers in the past such as those scanning heavily against SSH. Now I want to start considering emulating the complaint notification one can get from using fail2ban. So let’s start with just adding a simple external command lookup for getting the abuse contact for a given IP address. We will actually use the method found in the fail2ban complain module. So big thanks to them!

We want to have a search like this:
tag=authentication action=failure | stats count values(user) by src_ip | lookup abuseLookup ip AS src_ip

Once you add the transforms and python script below the command should work in Splunk. Keep in mind like the dnsLookup this has to happen on any search heads that will need it. I also have not yet worked on making this handle ipv6 which abusix.com can do with the lookups. The new abuseLookup will return a field to your events called abusecontact. Then you can use that how you want in reporting events.

First edit your transforms.conf to add this stanza:

Now create the python script abuseLookup.py in $SPLUNK_HOME/etc/system/bin/

Splunk Alert Scripts – Automating Control

A big thanks to the members of the @SplunkDev team that were helpful and patient with my questions while I pulled this together. Thanks Guys: @gblock, @damiendallimore‎ and David Noble

In Splunk circles, you often hear about the holy grail of using Splunk to actively control other systems. It can be hard to find details or good examples on HOW to do it. I am always working on something new that deepens my technical skills. I had not previously dealt with REST APIs or Splunk alert scripts and this post is the result. Used well you can replace manual daily operations tasks; changing Splunk from a tool into a team member.

We will cover a working example of using Splunk alert results to update a Google Spreadsheet via the Drive Python SDK. Once you understand how it works, you can make you own controls of any system that supports REST API calls such as an Intrusion Prevention System to block a list of IP addresses using a scheduled Splunk alert.

We will leverage a Splunk blog post on saving credentials in a Splunk App to avoid leaving our Google credentials hard coded and exposed in the alert script. It turns out alert scripts work in the same way but it is not well documented. I built a Python class for retrieving those credentials from Splunk so you could re-use the code across many alert scripts. The scripts can all be found in the supporting GitHub repo. You will be able to use these as a framework for your own alert scripts to drive actions in other systems. I will not be stepping through the code itself as it is fairly well commented. There are plenty of moving parts to this so you need to be an experienced Splunk administrator to get it working. The benefit is that once you get one working you can just make new variants with little effort.


Continue reading “Splunk Alert Scripts – Automating Control”

Splunk Presenting Data in statusboard on iPad

Splunk is a great tool for digging into data and presenting the results. Sometimes, you just want a status board of results that comes to you without having to log into a web application. A wonderful app for this is the iPad app statusboard by Panic software.

You always could create a panel on your statusboard that links to a URL of a file for presentation. However, this means your data is not protected by authentication. Panic added Dropbox support so you can now make a panel that pulls from a csv or json file. You can also airplay to an AppleTV or direct connect the iPad to a TV to present the dashboard on a large display.

In this post I will cover how I combined a Splunk alert script in python, dropbox and statusboard to get the result below. I am displaying the number of failed login attempts against my wordpress blog by country code for the previous 7 days. Keep in mind this is a Splunk instance running on my laptop with minimally sensitive information. I would never run dropbox directly on a work related production Splunk server. An alternative method would be to run a scheduled script that pulls the results out of Splunk via the REST api and write it out to a csv in the dropbox folder. I will do that version of this post in the future.

WordPress Logins Statusboard

Continue reading “Splunk Presenting Data in statusboard on iPad”

Splunk Alert Script – OSX Notification Center

I want to start making some custom alert scripts. As usual, I like to practice by using a live example. I have SSH remote access and Apache enabled on my laptop. When at work I keep a map up in Splunk on my laptop showing the source ip location of any attempts to connect to my laptop. If you start beating on my laptop it results in an instant ban hammer in the network IPS.

I sometimes miss seeing the map updates when busy. If I had an alert history that is quickly accessible it would be easier to handle the scanning systems. I decided on this alert to test the hits on apache that runs every 15 minutes. These logs just happen to go into an index called os_osx. I tagged the combined_access source type as “web”.

index=os_osx tag=web | stats count by clientip

Now the fun part. I am working on my python skills so I did the alert script in python. This required me to call the OSX shell command osascript in order to execute the Apple Script that generates the actual Notification Center message. It took a minute of experimentation to get the right combination of escaped quotes to build the Apple Script command.

We get a result like this:


And here is the alert script that I saved as osx-alert.py in the /Applications/splunk/bin/scripts folder on my laptop. That is the script I chose to call on the search above when saved as an alert.

OSX and Public Wifi – Toggle settings with AppleScript

I like to remind folks when moving their Apple laptop to public wifi that they need to remember to turn off the iLife application sharing such as iTunes and iPhoto. Then turn on their firewall.

So here is an AppleScript that will do just that. It is written and tested on OSX Lion with iLife 11. So you may have to play with it for your version if that is not what you are running.  Keep in mind it is a toggle script.  It will reverse the settings of iPhoto, iTunes sharing and the firewall. So it is assumed you share both with the firewall off when at home.

Also you need to ensure Enable access for assistive devices is checked under Universal Access in System Preference.

Just cut and paste the below script into the AppleScript editor.  Then save either as an application on your desktop you can double click. Or save as an AppleScript where an application like LaunchBar can use it as an action. ~/Library/Application Support/LaunchBar/Actions

Continue reading “OSX and Public Wifi – Toggle settings with AppleScript”

AppleScript- Getting Brandon Sanderson Project Updates read aloud.

I have been playing around making a Jarvis like home notifaction script.  One piece of that for fun is to read me the current Brandon Sanderson book project status report.  I made the below applescript to parse the information off his Web site and read it aloud to me. After all men don’t read (a Stormlight Archive reference so go get the book). I am a huge fan of his Mistborn and Stormlight Archive series.  Oh yeah… He is also finishing the Wheel of Time too. ;-)

Warning that the below script could break at any time.  It has to parse out bits of html code.  Who knows how much that changes each time updates his status.  I won’t know till the next update.  You can see those bits in the AppleScript’s text item delimeters lines in the script.




Posted via email from georgestarcher.randomlings