Orca Security event ingestion pipeline in Python, deployed with Puppet
Turning noisy cloud security alerts into events analysts can actually use.
In plain terms
Orca Security is a platform that scans a company's cloud environment and flags security issues. Some clients use it, and they needed those events in the SIEM: the central system where a client's security events come together so they can be watched in one place.
The problem was the shape of the data. Each Orca event arrives packed with nested detail, lists inside lists, and a lot of it isn't useful for monitoring. It had to be trimmed and flattened before it was useful.
Puppet, which comes into this later, is the tool we use to set up and manage servers from code, so a change can be rolled out the same way to many machines.
The problem
Clients using Orca Security needed its events in the SIEM, but each event carried a huge amount of nested data, so it had to be trimmed before it was useful.
What I did
I wrote a Python script that pulls events from the Orca Security platform and runs them through a full pipeline: no duplicates are sent, the data is flattened, and unneeded parent and child fields are removed.
Once the script was right, I deployed it with Puppet, with the secrets encrypted.
The outcome
The module runs in production for three clients, including an energy company, and is the standard for future deployments.