De-Coder’s Ring

Software doesn’t have to be hard

Month: June 2017

Simple Tip: Provision an Elasticsearch Node Automatically!

I built out a new Elasticsearch 5.4 cluster today.

Typically, it’s a tedious task.   I haven’t invested in any sort of infrastructure automation technology, because, well, there aren’t enough hours in the day.  I remembered a trick a few of us came up with at a previous bank I used to work for.  Using a shell script in AWS S3, that gets downloaded in a user init script in EC2, and bam, off to the races!

I won’t give away any tricks here, since my boss would kick me… again, but, since this processed was used heavily by me and team previously, I don’t mind sharing.

We didn’t use it specifically for Elasticsearch, but, you can get the gist of how to use it in other applications.

First step, upload the script to AWS S3.    Here, I’ll use an example bucket of “notmybucket.com” – that’s my bucket, don’t try to own it.  for reals.

Let’s call the script “provision.es.sh”

The provision file can look something like this:

You’ll see reference to an elasticsearch.data.yml.template.. that’s super simple:

Made up a security group, etc… configure the security group to whatever you’re using for your ES cluster.. change the bucket to your bucket.

Each ES host needs a unique name (beats me what will happen to elasticsearch if you have multiple nodes with the same name.. they’re geniuses.. it’s probably fine, but, you can test it, not me).  Alternatively, try to use your instance ID as your node name!

Then your user init data looks super stupid and simple:

Add user data

Once you complete your EC2 creation, you can verify the output in:

/var/log/cloud-init-output.log

 

Spilling the beans…

Today, I’m at ILTA’s LegalSec Summit 2017.    Giving a talk later about Threat Automation:

Me

Talk

I’m excited about sharing some information about Threat Intelligence, automation and application on a network sensor.  That’s all good stuff.

What I’m really happy about, is that I can be totally open with the technology.  My goal is to educate folks on how they can do what my company does on a daily basis.  As an open source advocate, and a giant fan of a lot of the technology that I use every day (duh!), it’s good to show others how to do it.  We don’t provide anything that qualifies as super cool intellectual property….  we have some, but anyone can build the basics to run in their shop.  The challenge comes with the human capital needed to build and run this stuff.   That’s a big part of the challenge.

© 2017 De-Coder’s Ring

Theme by Anders NorenUp ↑