ARCSIGHT INTERGRATION WITH ELASTICSEARCH USING ARCSIGHT SMARTCONNECTOR

The Elastic Stack is certified by ArcSight to support CEF-formatted data, whether generated by ArcSight or external sources. Stream events via ArcSight Smart Connectors or hook into the ArcSight Data Platform (ADP). Parsing and enrichment by Logstash simplify the indexing of data into Elasticsearch. Pre-built dashboards can be tailored to meet your specific needs.

Prerequisites

These step to step guide assumes that Logstash, Elasticsearch, and Kibana are already installed. The products you need are available to download and easy to install. The Elastic Stack 5.6 (or later) and X-Pack are required for this module. If you are using the Elastic Stack 6.2 and earlier, please see the instructions for those versions.

Note: The guide has Elasticsearch secured (However this can be achieved without Elasticsearch secured)

Deployment Architecture

The Logstash ArcSight module understands CEF (Common Event Format), and can accept, enrich, and index these events for analysis on the Elastic Stack. ADP contains two core data collection components for data streaming:

The Smart Connectors (SC) are edge log collectors that parse and normalize data to CEF prior to publishing to the Logstash receiver.

Install ArcSight Smart connector

Smart Connector has been configured to publish ArcSight data (to TCP port 5000) using the CEF syslog destination

1. Select the location to install smart-connector

2. Select Program group

Continue to install

3. Add connector

4. Choose connector to configure

5. Select CEF syslog as type of destination

6. Specify the IP/host of the Elasticsearch destination and port number (5000), Protocol (Raw TCP)

For a successful connection from the smart connector to Logstash, Logstash service need to be running

Logstash Configuration — ArcSight Setup

1. Edit config/logstash.yml

modules:
- name: arcsight
  var.inputs: smartconnector
  var.input.smartconnector.port: 5000
  var.elasticsearch.hosts: http://127.0.0.1:9200
  var.elasticsearch.username: elastic
  var.elasticsearch.password: changeme
  var.kibana.host: roscigno-demo:5601
  var.kibana.username: elastic
  var.kibana.password: changeme
  var.elasticsearch.ssl.enabled: false
  var.kibana.scheme: http
  var.kibana.ssl.enabled: false
  var.kibana.ssl.verification_mode: disable
./bin/logstash --modules arcsight --setup
3.       Run Logstash listening for ArcSight connections
./bin/logstash --modules arcsight
In order to get data flowing change Directory to
changes:
type => _doc
and
output {
<%= elasticsearch_output_config('_doc') %>

Indexing incoming events on Kibana

Define the index pattern

View events

Reference

https://github.com/elastic/logstash/issues/10613

https://www.elastic.co/guide/en/logstash/5.6/arcsight-module.html

https://www.elastic.co/guide/en/elastic-stack/current/installing-elastic-stack.html