De-Coder’s Ring

Consumable Security and Technology

Page 3 of 21

Routing Messages through Kafka

I’m going through a major project with a client in regards to migrating to a Kafka based streaming message platform.  This is a heckuva lot of fun.   Previously, I’ve written about how Kafka made the message bus dumb, on purpose:   find it here. 

Building out a  new streaming platform is an interesting challenge.  There are so many ways to handle the logic.  I’ll provide more details as time goes on, but there are at least three ways of dealing with a stream of data.

The Hard Wired Bus

A message bus needs various publishers and subscribers.  You can very tightly couple each service by having them be aware of what’s upstream or what’s downstream.  Upstream is where the message came from, downstream is where it goes next.  When each component is aware of the route a message must take, it becomes brittle and hard to change over time.  Imagine spaghetti.

The Smart Conductor

A conductor travels on the train.  The conductor determines the best route while moving along the train tracks.  The conductor can handle every message, after every service to determine which service is next in line.  This cleans up the function of each service along the line but makes the conductor pretty brittle too.  The more complex the system gets, the more complex the conductor gets.  A rules engine would be a great component to add to the conductor if you choose this path.

The Map Maker

A map maker plots a route for a hiker to follow.  In our case, the map maker is a component that sits at the very beginning of every stream.  When the event comes to the map makers topic (in Kafka), the map maker determines the best route for the message to take.  Using metadata or embedded data in the event, the map maker can send the route down the chain with the event itself.  Each service can use a common library to read from its configured topic, allow the custom service to do some work, and then use the wrapper again to pass the message downstream.  The biggest advantage here is that each service doesn’t care where the message comes from, or where it goes next.   This works great for streams that are static, and the route can be determined up front.    If there are decisions down stream, then it may need a ‘switch’ service that is allowed to update the route.

What’s the best path for your application?    Have something better that I haven’t thought of yet?

 

 

 

 

 

Podcast – John Lockie on Tech (part 2)

This week is another great interview with John Lockie.. well, really, it’s a continuation of last week’s that you can find here:  Podcast – John Lockie Interview (part one)

John goes straight to some solutions for networks (capture all the things) as well as credential monitoring (“they pop shell ridiculously easy”).

Listen to the knowledge!

Walking on Water Referral

ThumbsUp

ThumbsUp

I had a college professor, Dr David Bernstein, once talk about recommendations and referrals. The line he said has stuck with me for near 20 years since I heard it:

“Don’t give a reference for someone if you can’t, by all good faith, make them sound like they walk on water.” – Dr David Bernstein

Over the past 20 years of my career, I’ve had dozens, if not hundreds of people ask me for a recommendation. Whether they’re going for a new job, a security clearance or some sort of promotion, it’s the first thing they need to iron out. They need solid references so there is more trust built around their case.

This seems to make a lot of sense, but, it’s really hard to tell someone no. You can wuss out and say “oh, my company policy won’t allow me to provide a referral”. Ok, that really may be the case, but, your integrity is important. Sometimes, we need to tell people the truth in order to help them grow.

If you have to say no, let them know why. No need to be mean, but be constructive. If they push you for “why” you can’t give a recommendation, talk to them about specific incidents or habits that they could improve upon.

If they have done something in the past to break your trust, and they still ask you, then you can laugh at them. That’s a no go. Protect your integrity at all costs.

Finding Solid Information

Ever find an eye-opening new source of information? Not technical information, like javadocs (are they still a thing?), but personal growth information.

in the old days, we had technical sources like slashdot, freshmeat, digg  etc.. but, times have gone on, and now we have other places we can read regularly to keep up… here are some links and reasons why I love them:

Web Sites
https://www.reddit.com/r/programming/ – Lots of technical stuff.  New, updated code, procedures, standards, etc.

www.reddit.com/r/startups – I have a great deal of passion around new companies and people getting up, killing it and dragging it home.

https://techcrunch.com/ – Tech Crunch is great for big industry news, and a nice place to find information on up and comers

https://news.ycombinator.com/ – Hacker News – Great place for anything tech related, up and coming

Podcasts

Civics 101 – Not tech, but, crazy informative and a must listen for any US citizen and anyone who wants to learn how the US works.

http://www.npr.org/podcasts/512508710/civics-101

Science Friday – Cause, Science

https://www.sciencefriday.com/listen/

Source Code Podcast – This is new for me, and fantastic:

Podcast

 

What am I missing out on?!

Podcast – John Lockie Interview (part one)

Over the next few months, I plan on doing a big podcast binge on cybersecurity careers and will continue my focus on technology.

This week’s episode, John Lockie and I talk about his background and how it’s not the traditional path into cybersecurity if there really is one.  He affirms my beliefs in regards to CISOs with music degrees.  You’ll never guess what he says!

Find John on Twitter:  @thedefensedude

Please find and subscribe to the De-Coder’s Ring” in iTunes, and I’d be ecstatic if you gave me a great rating.

Leave feedback below about this episode!

 

« Older posts Newer posts »

© 2018 De-Coder’s Ring

Theme by Anders NorenUp ↑