Thursday, 20 November 2014

QRadar Architecture

QRadar Architecture

Today I was reading about the new QRadar integration with the IBM BigData solution. Instead of writing down here, I decided to share with you guys a very nice video that summarize the benefits of this integration.

Centralized vs. Distributed collecting


One of the main questions when designing the architecture of a QRadar environment is using a centralized (with or without clustering) or a distributed deployment. It means, should we create a cluster of QRadar in a specific network or should we distribute our collectors across the networks? As usual, the answer is: Depends.
The following pictures summarize the benefits and cons of the both cases.
Centralized
Centralized Deployment
In the Centralized scenario, all the servers and collectors are in the same network. It makes the deployment and management way easier since we have just one point of maintenance and one point to “care about”, and it is very important especially when we have a geographically spread environment. But having all the SIEM solution in one network means that all the environment will need to connect to the cluster. In other words, the firewalls will allow traffic between the QRadar cluster and any server. Considering that some collection methods involves windows authentications, it means that if someone get access to the QRadar cluster network, the person will have access to any device on the network.  Another bad point of this kind of deployment is the network failure tolerance. Lets say that the router in the border of the QRadar network goes down, all the log collection will be lost.

Distributed
Distributed Deployment

The distributed collection usually takes more time (and money) to implement and requires more time/resources to maintain, since the appliances will be distributed physically and logically. But the advantages are clear. With a distributed deployment the main QRadar console will have access only to its’ collectors, and nothing more. It means that if someone get access to the main SIEM network, the person will be able only to send packets to very specific IPs (collectors), and since the QRadar collectors are completely hardened, the security risk involved on this deployment is very low. Another benefit of the distributed deployment is the network failure tolerance. Considering the same case of a broken router in the QRadar console network, in this case the collectors will not have connection with the main console and will buffer the logs. After the network connectivity being restored, the logs will be synchronized with the main console.
As you guys noticed, the Distributed deployment can bring some good advantages compared with the Centralized one. But each company is a different case. Is up to you as an architect decide which deployment will fit your client need.
Do you have any suggestion or comment? Drop us a line in the comments!

Qradar

Qradar

IBM® QRadar® Security Intelligence Platform products provide a unified architecture for integrating security information and event management (SIEM), log management, anomaly detection, incident forensics and configuration and vulnerability management. These products offer advanced threat detection, greater ease of use and lower total cost of ownership.
IBM QRadar Security Intelligence Platform products deliver:

Product editions:

IBM Security QRadar Incident Forensics: Allows you to retrace the step-by-step actions of a potential attacker, and quickly and easily conduct an in-depth forensics investigation of suspected malicious network security incidents.
Features | View prices
IBM Security QRadar Log Manager: Delivers high-performance for collecting, analyzing, archiving and storing large volumes of network and security event logs.
Features | View prices | Product support (US)
IBM Security QRadar Network Anomaly Detection: Enhances IBM intrusion prevention system (IPS) solutions by providing greater insight into network behavior and abnormal activity to better identify security threats.
Features | Product support (US)
IBM Security QRadar QFlow Collector: Combines with IBM Security QRadar SIEM and flow processors to provide Layer 7 application visibility and flow analysis, helping you understand and respond to activities throughout your network.
Features | View prices | Product support (US)
IBM Security QRadar Risk Manager: IBM Security QRadar Risk Manager monitors network topology, switch, router, firewall and Intrusion Prevention System (IPS) configurations to reduce risk and increase compliance.
Features | View prices | Product support (US)
IBM Security QRadar SIEM: Consolidates log source event data from thousands of devices endpoints and applications distributed throughout a network. It performs immediate normalization and correlation activities on raw data to distinguish real threats from false positives.
Features | View prices | Product support (US)
IBM Security QRadar VFlow Collector: Combines with IBM Security QRadar SIEM to provide Layer 7 application-layer visibility into virtual network traffic, helping you understand and respond to activities in your network.
Features | View prices | Product support (US)
IBM Security QRadar Vulnerability Manager: IBM Security QRadar Vulnerability Manager proactively discovers network device and application security vulnerabilities, adds context and supports the prioritization of remediation and mitigation activities.

Configuring Snort with Barnyard, Snort Report, Acid in Ubuntu 14.04



Main features introduced in 2.9.6: 
  •  Feature rich IPS mode including improvements to Stream for inline deployments. Additionally a common active response API is used for all packet responses, including those from Stream, Respond, or React. A new response module, respond3, supports the syntax of both resp & resp2, including strafing for passive deployments. When Snort is deployed inline, a new preprocessor has been added to handle packet normalization to allow Snort to interpret a packet the same way as the receiving host. 
  • Use of a Data Acquisition API (DAQ) that supports many different packet access methods including libpcap, netfilterq, IPFW, and afpacket. For libpcap, version 1.0 or higher is now required. The DAQ library can be updated independently from Snort and is a separate module that Snort links. See README.daq for details on using Snort and the new DAQ.
  • Updates to HTTP Inspect to extract and log IP addresses from X-Forward-For and True-Client-IP header fields when Snort generates events on HTTP traffic.
  • A new rule option ‘byte_extract’ that allows extracted values to be used in subsequent rule options for isdataat, byte_test, byte_jump, and content distance/within/depth/offset. 
  •   Updates to SMTP preprocessor to support MIME attachment decoding across multiple packets. 
  •  Ability to “test” drop rules using Inline Test Mode. Snort will indicate a packet would have been dropped in the unified2 or console event log if policy mode was set to inline. 
  •  Two new rule options to support base64 decoding of certain pieces of data and inspection of the base64 data via subsequent rule options. 
  •  Updates to the Snort packet decoders for IPv6 for improvements to anomaly detection.
A few days ago a new version of Snort was released, in this version some things about compiling  have slightly changed, the libdnet and the Data AcQuisition library (DAQ) must be compiled separately. In this post I’m going only to illustrate how to compile and install Snort 2.9.6 from the source code. 
LAMP environment:
Install Apache, PHP and MySQL: 
#apt-get -y install apache2 libapache2-mod-php5 mysql-server mysql-common mysql-client php5-mysql libmysqlclient-dev php5-gd php-pear libphp-adodb php5-cliScreenshot from 2014-05-02 16:57:08Needed packages:
#apt-get -y install libwww-perl libnet1 libnet1-dev libpcre3 libpcre3-dev autoconf libcrypt-ssleay-perl libtool libssl-dev build-essential automake gcc make flex bisonScreenshot from 2014-05-02 16:56:24Download and Install libdnet:
There are Ubuntu packages for libdnet but this is an easier method of installation. Download the following file and install it with these commands from your download directory:
#mkdir /usr/local/snort
#cd /usr/local/snort
#tar xzvf libdnet-1.12.tgz
#cd libdnet-1.12/
#./configure
#make
#make install
#ln -s /usr/local/lib/libdnet.1.0.1 /usr/lib/libdnet.1Screenshot from 2014-05-02 17:06:56Installing and Downloading DAQ:
Snort 2.9.0 introduces the new Data Acquisition API. We’ll need to download and install it before we set up the core Snort package.
Download that package to your Snort machine and install it using the following commands:
#cd /usr/local/snort
#tar zxvf daq-2.0.2.tar.gz
#cd daq-2.0.2
#./configure
#make
#make installScreenshot from 2014-05-02 17:03:01Download and Install libpcap:
#cd /usr/local/snort 
#cd libpcap-1.3.0
#./configure
#make
#make install
#echo “/usr/local/lib” >> /etc/ld.so.conf
#ldconfig -vScreenshot from 2014-05-02 17:15:25Download and Install Snort:
While we could install the Snort packages from the Ubuntu 12.04 repositories, that doesn’t guarantee the latest and greatest version of Snort being set up so we’re going to compile and install the source code. Openhttp://www.snort.org/snort-downloads with your browser and download the newest stable version.
The following steps will install Snort into /usr/local/snort but you can change this to a directory of your liking by modifying the paths below. 
Open a command prompt and issue the following commands from the directory where you downloaded the Snort 
#tar zxf snort-2.9.6.1.tar.gz
#cd snort-2.9.6.1
#./configure –prefix=/usr/local/snort –enable-sourcefire
#make
#make install
#mkdir /var/log/snort
#mkdir /var/snort
#groupadd snort
#useradd -g snort snort
#chown snort:snort /var/log/snortScreenshot from 2014-05-02 17:18:09Download the Latest Snort Rules:
The next step is to download the latest Snort ruleset. You’ll need to log into the Sourcefire site in a browser in order to get the file. The latest rules are located here: https://www.snort.org/snort-rules.
There are two sections on this page – one for VRT subscribers and one for registered users. The only difference is that the registered user rule files are 30 days older than those for subscribers.
Download this file to your IDS machine: snortrules-snapshot-2960.tar.gz. 
Open a command prompt in the directory where you downloaded the Snort ruleset file and issue the following commands: 
#tar zxf snortrules-snapshot-2960.tar.gz -C /usr/local/snort
#mkdir /usr/local/snort/lib/snort_dynamicrules
#cp /usr/local/snort/so_rules/precompiled/Ubuntu-12-4/x86-64/2.9.5.3/* /usr/local/snort/lib/snort_dynamicrules
#touch /usr/local/snort/rules/white_list.rules
#touch /usr/local/snort/rules/black_list.rules
#ldconfig -vScreenshot from 2014-05-02 17:55:21Now we need to edit the snort.conf configuration file:
#vi /usr/local/snort/etc/snort.conf 
var WHITE_LIST_PATH /usr/local/snort/rules
var BLACK_LIST_PATH /usr/local/snort/rules 
dynamicpreprocessor directory /usr/local/snort/lib/snort_dynamicpreprocessor/
dynamicengine /usr/local/snort/lib/snort_dynamicengine/libsf_engine.so
dynamicdetection directory /usr/local/snort/lib/snort_dynamicrules 
#output unified2: filename merged.log, limit 128, nostamp, mpls_event_types, vlan_event_types
output unified2: filename snort.u2, limit 128
:wq!Screenshot from 2014-05-02 17:56:26
Screenshot from 2014-05-02 17:57:25Download and Install Barnyard2:
Barnyard2 improves the efficiency of Snort by reducing the load on the main detection engine. It reads Snort’s unified logging output files and enters them into a database. If the database is unavailable Barnyard will input all data when the database comes back online so no alerts will be lost.
#cd barnyard2
#autoreconf -fvi -I ./m4
#./configure –with-mysql –with-mysql-libraries=/usr/lib/x86_64-linux-gnu
#make
#make install
#cp etc/barnyard2.conf /usr/local/snort/etc
#mkdir /var/log/barnyard2
#chmod 666 /var/log/barnyard2
#touch /var/log/snort/barnyard2.waldo
#chown snort.snort /var/log/snort/barnyard2.waldoScreenshot from 2014-05-02 18:02:44
Screenshot from 2014-05-02 18:04:03let’s create the MySQL database and the database schema. You’ll have to enter the MySQL password that you chose earlier in the next two steps:
#echo “create database snort;” | mysql -u root -p
#mysql -u root -p -D snort < ./schemas/create_mysqlScreenshot from 2014-05-02 18:06:08Next we need to create an additional MySQL user for Snort to use as it’s not a good idea to run the daemon as root. Remember the password that you enter below. Also note the single quotes around the password in addition to the double quotes around the entire echo statement:
#echo “grant create, insert, select, delete, update on snort.* to snort@localhost identified by ‘bhuvi’” | mysql -u root -pScreenshot from 2014-05-02 18:08:08Modify the Barnyard2 configuration file with the following command:
#vi /usr/local/snort/etc/barnyard2.conf
config  reference_file: /usr/local/snort/etc/reference.config
config  classification_file: /usr/local/snort/etc/classification.config
config  gen_file: /usr/local/snort/etc/gen-msg.map
config  sid_file: /usr/local/snort/etc/sid-msg.map
config hostname: localhost
config interface: eth0
output database: log, mysql,Screenshot from 2014-05-02 18:11:35Testing Snort:
You can test to see if Snort will run by using this command: 
#/usr/local/snort/bin/snort -u snort -g snort -c /usr/local/snort/etc/snort.conf -i eth0Screenshot from 2014-05-03 09:13:35You should see a message saying “Commencing packet processing.” You can cancel out of it by hitting Control-C. If it fails to initialize please see the forums at snort.org to determine the problem. It will usually be something in the configuration file.
To set Snort to start automatically on your machine edit the rc.local file with the following command:
#vi /etc/rc.local
ifconfig eth0 up
/usr/local/snort/bin/snort -D -u snort -g snort -c /usr/local/snort/etc/snort.conf -i eth0 /usr/local/bin/barnyard2 -c /usr/local/snort/etc/barnyard2.conf -d /var/log/snort -f snort.u2 -w /var/log/snort/barnyard2.waldo -D 
:wq!Screenshot from 2014-05-03 09:22:33Save the file and exit. Then either reboot or use the following command to start Snort:#/etc/init.d/rc.local start
Monitoring Your System: 
Download and Set up Snort Report
The next step is to download and configure Snort Report. It’s available athttp://www.symmetrixtech.com under the downloads section.
As of this writing the current version is 1.3.4. Download snortreport-1.3.4.tar.gz to a directory on your IDS machine.Open a command prompt in the directory to which you downloaded Snort Report and issue the following 
#tar zxvf snortreport-1.3.4.tar.gz -C /var/www/html 
Now we need to modify the Snort Report configuration file to reflect your MySQL login info and location of the jpgraph libraries. Change the file by editing srconf.php with this command: 
#vi /var/www/snortreport-1.3.4/srconf.php 
$pass = “bhuvi”;Screenshot from 2014-05-02 17:02:04Install JPGraph:
#cd /var/www/html
#tar xvzf jpgraph-3.5.0b1.tar.gz
#rm -rf jpgraph-3.5.0b1.tar.gzScreenshot from 2014-05-01 13:33:38Installing ADODB:
#cd /var/www/html
#wget  http://kaz.dl.sourceforge.net/project/adodb/adodb-php5-only/adodb-518-for-php5/adodb518a.tgz
#tar xzf adodb518a.tgz
#rm adodb518a.tgzScreenshot from 2014-05-01 13:35:12Installing and configuring Acid:
#cd /var/www/html
#wget  http://acidlab.sourceforge.net/acid-0.9.6b23.tar.gz
#tar xzf acid-0.9.6b23.tar.gz
#rm acid-0.9.6b23.tar.gz

Screenshot from 2014-05-01 13:34:12
#vi /var/www/html/acid/acid_conf.php
$DBlib_path = “/var/www/html/adodb518a”;
$alert_dbname = “snort”;
$alert_host = “localhost”;
$alert_port = “”;
$alert_user = “snort”;
$alert_password = “bhuvi”; 
$archive_dbname = “snort”;
$archive_host = “localhost”;
$archive_port = “”;
$archive_user = “snort”;
$archive_password = “bhuvi”; 
$ChartLib_path = “/var/www/html/jpgraph-3.5.0b1/src”; 
:wq!Screenshot from 2014-05-02 10:27:52Start Apache then go to http://yourhost/acid/acid_main.php . You will get a message that looks like this in your browser:Screenshot from 2014-05-03 09:32:51Then click the button that says “Create Acid AG”Screenshot from 2014-05-03 09:33:01That’s it now browse to acid main page , it will show record details of snort…