Forensics folks have been doing this for years during investigations, but proactive continuous full packet capture – for the inevitable incident responses which haven’t even started yet – is still an early market. That’s a start, but you will likely require some kind of Big Data thing, which should be clear after we discuss what we need this detection platform to do.
We spent a time early in this process on sizing up the adversary for some insight into what is likely to be attacked, and perhaps even how. But once you do the work to model the likely attacks on your key information, and then enumerate those attack patterns in your tool, you can get tremendous value.
We have already listed a number of different threat intelligence feeds, which can be used to search for specific malware files, command and control traffic, DNS request patterns, and a variety of other indicators.
So you can search your security data infrastructure for almost anything you are collecting – or even better, for a series of events and/or files within your environment – quickly and accurately to narrow down your searches to the most likely attacks.
We have every confidence that big data holds promise for security intelligence, both because we have witnessed attacker behavior captured in event data just waiting to be pulled out, and because we have also seen miraculous ideas sprout from people just playing around with database queries.
You are clearly constrained in terms of internal capabilities (you will be looking for a lot of data scientists over the next few years), as well as the lack of maturity of technologies such as Hadoop, MapReduce, Pig, Hive, and a variety of others in the security context.
But companies seriously looking to detect advanced attackers within their environments will be capturing packets to supplement the other data they already collect, and subsequently starting to use Big Data technologies to mine it all.
Link: https://securosis.com/blog/the-cisos-guide-to-advanced-attackers-mining-for-indicators