Nov 13, 2015

Process NetFlow with nProbe and Elasticsearch, Logstash, and Kibana - Part 1

Install Elasticsearch, Logstash, and Kibana on Windows Server 2012 R2


By analyzing the data provided by NetFlow, a network administrator can determine things such as the source and destination of traffic, class of service, and the causes of congestion. A typical flow monitoring setup (using NetFlow) consists of three main components:
  • Flow exporter: aggregates packets into flows and exports flow records towards one or more flow collectors.
  • Flow collector: responsible for reception, storage and pre-processing of flow data received from a flow exporter.
  • Analysis application: analyzes received flow data in the context of intrusion detection or traffic profiling, for example.
In this tutorial, we will use:
  • NetFlow generator ( as flow exporter
  • nProbe ( as flow collector
  • Elasticsearch + Logstash + Kibana (ELK to receive, store, analyze, and display Netflow data
    System Diagram
    The diagram above shows how Netflow data are processed.

    A simple network diagram is created for this tutorial
    Network Diagram
    ELK and nProbe will be installed on, and sample NetFlow data will be generated from

    Let's start by setting up an ELK stack on Windows Server 2012 R2

    Ref: I eliminate some extra steps to simplify the tutorial.

    Install Prerequisites

    1. Download and install latest JRE from Oracle (currently jre-8u66-windows-x64.exe). You may install either JDK or JRE.
    2. Add JAVA_HOME environment variable. Open System Properties

    Create a new system variable

    Set JRE folder path, change this path if you update JRE later

    Install ELK

    1. Download ELK for Windows from
    2. Create a folder for ELK at C:\ELK, extract all ELK packages
    3. Open Command Prompt and change to elasticsearch\bin folder (Tip: You can use Shift + Right Click on a folder to open Command Prompt at that folder)

    To install Elasticsearch, run
    service install

    If you plan to install more than one Elasticsearch instance on the same server, edit the file elasticsearch\bin\service.bat and change
    set SERVICE_ID=elasticsearch-service-x64
    set SERVICE_ID=elasticsearch-service-x64-<custom-name>
    Open Elasticsearch service manager, start Elasticsearch service
    service manager
    Update path to new JRE if necessary
    Start service

    Open a web browser, go to http://localhost:9200. We should see as the screenshot below

    4. NSSM ( is used to install and run both Logstash and Kibana as a Windows service
    Extract and copy file nssm.exe from nssm-2.24\win64 to

    5. To install Kibana, open Command Prompt at C:\ELK\kibana\bin and run
    nssm install kibana

    Locate kibana.bat

    Install service

    Go to http://localhost:5601 to see if Kibana is working

    6. To install Logstash, 
    Go to C:\ELK\Logstash\bin, create a Logstash config file named logstash.conf and paste the following lines
    input {
    tcp {
    port => 5544
      type => "netflow"

    filter {
    json {
    source => "message"

    output {
    stdout { 
    codec => rubydebug 
    elasticsearch {
    hosts => ["localhost:9200"]
    index => "netflow-%{+YYYY.MM.dd}"

    Create a run.bat file in the same folder and paste:
    logstash.bat agent -f logstash.conf
    Open Command Prompt at C:\ELK\logstash\bin and run
    nssm install logstash

    Locate run.bat

    Start service
    Check if Logstash is listening on port 5544


    Our ELK stack is ready to receive NetFlow and other data. In the next part, we will install nProbe, generate NetFlow data, and play with Kibana.
    Part 2: