De-Coder’s Ring

Consumable Security and Technology

Month: March 2013

Network Traffic Visualization – Where’s my traffic?

Where’s my network traffic?

Using Graphite and nPulse CPX API to map out the network

It was late.   The lab administrators were gone, and I work 95 miles away from our data center.  At work, we’re working on setting up a new and improved QA/Testing rack of equipment, and I was trying to run my automated tests.   Unfortunately, I misread a memo, and didn’t know where the data was going.

For our testing purposes, we have a custom replay appliance that exposes its operations via a RESTful API.  Our CPX platform does as well, more on that in a second.  So, when I passed some commands to the replay box, I didn’t get the data I expected.  I tried again.  Nothing.  Hmmm.. No one was there to help troubleshoot, so, I had to figure it out, remotely.

Tools

The newest tool is my tool chain is:
Graphite (Graphite Web, Carbon, Whisper).  http://goo.gl/UnqbN
combined with our CPX platform:  http://goo.gl/qqnLx
old faithful, Tornado’s HTTP Client: http://goo.gl/O4kHH

Process

I have access to a bunch of machine in our development and test lab, so that helps.  Using ‘my’ general virtual machine (Debian 6 Linux), I set up a graphite-web installtion.  More on that later. It’s kind of a bear to get installed on Debian.

I whipped up a quick script that loop through our CPX boxes, to watch their stats.  We have  a pretty simple RESTFul API to get capture statistics.  The plan is to grab the stats, create some entries in the Whisper database and then watch a graph to see where the traffic spikes.   (From now on, I’m just going to use Graphite as the entire system. So, I will put data in Graphite.  Although really, the data goes to Carbon, which puts it in Whisper, which is then served and visualized by Graphite-Web)

The format of the data is:

name.spaced.attribute value timestamp

in python:

“%s %d %d” % (name, value, time)

The CPX Capture Statistics end point takes this format, this returns a JSON structure:

‘https://%s/api/channel/capture?polling=true’ % cpx[‘url’]

So, to set up my python array of CPXs,

Then, simply enough, I loop the CPXs, build my URL, make a tornado request, and get the data back.
Then I loop through the stats of interest, build the appropriate Graphite formatted string, append it to my buffer, then send it away.

Graphite Web

What I found to be the simplest way of charting exactly what I wanted to chart, was to use the ‘render’ API that graphite-web provides.  Essentially, it’s a URL that outputs a PNG based on parameters.  It even takes a wild card, so, in one fell swoop, I can get a PNG showing the total ‘mbps’ per CPX.

Looks like this, for our ‘steady state’ traffic.

CPX Traffic Monitoring Steady State

Not replaying any traffic from my source. These boxes have data from elsewhere.

Then, after doing some experimentation with our replay end point, I can watch the graphite charts, to see which CPX is getting traffic, based on different parameters. Pretty slick! Now, I know where my traffic is going!

CPX Traffic Monitoring Spikes

Each spike shows what CPX received traffic.


 

 

Footnote URLs:

http://graphite.readthedocs.org/en/0.9.10/overview.html#about-the-project

http://www.npulsetech.com/Products/HammerHead-Flow-Packet-Capture

http://www.tornadoweb.org/en/branch2.4/httpclient.html

Netflix – Poor use of Google Ads?

I searched consumerist.com and did a little Google-ing to see if I could find a reference, but couldn’t, so here goes.

One of the greatest movies of all times is “The Princess Bride”.  Look it up. Yes, it’s the best. 

The boys and I wanted to check it out the other day, I decided to figure out what the best way to stream it was.  We have a Netflix account, Hulu Plus, Apple TV, FiOS, etc… I started at Google, to see where to stream it:

Google search for 'Princess Bride Streaming' shows a paid ad for Netflix.

Google search for ‘Princess Bride Streaming’ shows a paid ad for Netflix.

 

Boom. Netflix.   Great, I think. Let’s go to old faitful Netflix, and watch it.  So, we get ready to watch, and it’s not streaming.   Umm.. Their paid for advertisement says I can stream it.

Netflix search, showing it's a real search, for "princess b"

Netflix search, showing it’s a real search, for “princess b”

Empty "Princess Br" search on Netflix.  No princess bride.

Empty “Princess Br” search on Netflix. No princess bride.

I’m sure it’s an automated Ad Words placement, and nothing nefarious, but this could be close to false advertising.

 

© 2017 De-Coder’s Ring

Theme by Anders NorenUp ↑