Splunk New Technology Add-ons: SyncKVStore and SendToHEC

I recently updated and replaced older repositories from my GitHub account that were hand made modular alerts to send search results to other Splunk instances. The first one sends the search results to a Splunk HTTP Event Collector receiver. The second one came from our Splunk 2016 .conf talk on KVStore. It was useful for sending search results (typically an inputlookup of a table) to a remote KVStore lookup table.


You can find the updated Send To HEC TA on Splunkbase: TA-Send_to_HEC or in my GitHub repository: TA-Send_to_HEC.

This is useful for taking search results and sending to another Splunk instance using HEC. If you chose JSON mode it will send the results as a JSON payload of all the fields after stripping any hidden fields. Hidden fields start with an underscore. RAW mode is a new option which takes the _raw field and sends ONLY that field to the remote HEC receiver.


This has been completely redone. I have submitted it to Splunkbase, but for the moment you can get it from my GitHub repository: TA-SyncKVStore

Originally it only sent search results to a remote KVStore. Now it also has two modular inputs. The first pulls a remote KVStore collection (table) and puts it into a local KVStore collection. The second pulls the remote KVStore collection but indexes it locally in JSON format. It will strip the hidden fields before forming the JSON payload to index. You are responsible for making sure all the appropriate and matching KVStore collections exist.

If you look in the code you will notice an unusual hybrid of the Splunk SDK for Python to handle KVStore actions and my own python class for batch saving the data to the collection. I could not get the batch_save method from the SDK to work at all. My own class already existed and was threaded for performance from my old version of the modular input so I just used the SDK to clear data if you wanted a replace option and then my own code for saving the new or updated data.

I rebuilt both of these TAs using the awesome Splunk Add-on Builder. This makes it easy in the SyncKVStore TA to store the credentials in the internal Splunk encrypted storage. One comment to update on the previous post on credential storage. The Add-on Builder was recently updated and now gives much better multiple credential management with a “global account” pull down selector you can use in your inputs and alert actions.

crowbar BMG/KC Alerts on your iPhone

One of the things I built into my crowbar dictionary attack tools for DMG and keychain files from the start was Growl.  Growl is a free add on notification framework for your mac. MANY popular mac programs support growl so this is not just some odd plug in.   I recommend in the crowbar apps making it popup the notifications for password found and not found at least go to your screen.  The password found is even better when set to sticky.  This means the alert stays on the screen until you click on it.

Now if you have an iPhone you can get the alert notifications right to your iPhone.  There is a great iPhone application called Prowl (App Store Link).  The developer’s site lets you create a login to his site which you set in the Prowl program.  You download and install a Growl plugin for Prowl.  The Prowl iPhone app is $2.99.  The service and plugin are free.  Last all you do is customize the alert settings for the crowbar apps to send to Prowl just using the growl preference pane control.

Now when you leave those real large dictionaries running you can leave them minimized and even leave the office or home knowing you will get the status when the job finishes.

You can find out everything at the Prowl developer’s site: http://prowl.weks.net/

crowbar Apps maintenance update 1.0.2

I dropped v1.0.2 of both crowbarDMG and crowbarKC into the automatic update feed.  Please just run the applications and choose Check for Updates or allow automatic updates to run.

This update fixes where I was not stripping the carriage return characters from windows CRLF formatted text files used as dictionaries.  It would cause the program to appear it was properly checking passwords but never find the correct password due to the extra CR character.

Mac and Sleuthkit

I love using Sleuthkit tools fls and mactime to produce a timeline for file system analysis.  But what if you are not compiler friendly and have a mac as your forensics workstation?  Here is the quick and easy way to get Sleuthkit installed so you can run it against raw disc images.

  1. Get macports from macports.org  It is a simple install from dmg.
  2. Once installed, get a terminal session opened.
  3. execute the command: sudo port -d selfupdate
  4. execute the command: sudo port install sleuthkit

It will take a while for sleuthkit and all the dependancies to install.  Once done you should be able to do “man fls” and “man mactime” to see the manual pages for the tools and start using them.

Mozy (on out of here) Backup for Mac

A long time back I had tested the online backup service Mozy.  By long time back I mean my version was mozy-0_6_2_6-502.dmg.  Today I was trouble shooting an application I am beta testing for someone.  I needed console logs.  Low and behold the Mozy removal script from that version was so bad it had left something behind.  I have TONS of the following events showing in my Console.

8/6/08 9:15:53 PM com.apple.launchd[1] (com.mozy.backup[1457]) posix_spawnp(“/Applications/Mozy.app/Contents/Resources/MozyBackup”, …): No such file or directory 

8/6/08 9:15:53 PM com.apple.launchd[1] (com.mozy.backup[1457]) Exited with exit code: 1 

Well a bit of googling and I find that this combination works to finally get rid of that sucker.

sudo launchctl unload /Library/LaunchDaemons/com.mozy.backup.plist

Follow that up with going into the /Library/LaunchDaemons and tossing the file com.mozy.backup.plist into the trash.  Now I have nice clean console logs for troubleshooting a real problem.  Not something sucking up CPU cycles trying to relaunch every 10 seconds.