Even white hat hackers have to deal with Big data these days if they want to be efficient and do their job right. But this is not always the easiest job and with utilizing right tools we can create amazing visualizations and look for juicy stuff in there. There will be demonstration, what kind of interesting data can be found with some simple setup.
When doing penetration tests we often run into big number of different data. One of those fields are also Wi-Fi networks and huge sizes of log files. When doing Wi-Fi analysis we are mostly focused on using aircrack-ng or Kismet toolset. This means, that we are generally limited to terminal windows and text outputs. This kind of data is hard to visualize and since humans can easily analyze data when there is good visual representation, there is place to do some research in this area. Same goes with all kinds of log files for example Apache or other web server logs, firewall logs, etc. And all this Big data can be really good handled with use of ELK stack.
ELK stack is a modern solution that can handle large amount of data and make it search and visualize in an easy way. ELK stack is composed from three different components. Logstash takes care for secure log storage transfer from clients to central repository. Elasticsearch is a search server based on Lucene. It provides a distributed, multitenant capable full-text search engine with a RESTful web interface and schema-free JSON documents. Kibana is virtualization package that can help better understand large volumes of data, easily create bar charts, line and scatter plots, histograms, pie charts, and maps and this without writing a single line of code.
Putting all these things together we are able to quickly create interesting dashboards, which can help to identify some patterns or even tell more about the mobile device users, attackers and potential threats. Also some things will be pointed out, how to contribute back to Open source community with your research work.
Outline of presentation:
Need for analyzing big data for hackers
How to fit this into whole ecosystem
ELK stack and how to build and use it
Two use cases how this can be used
Analyzing Wi-Fi packets – creating small NSA Wi-Fi station
Wrap up with some other potential use