Configure Stream forwarder

There are two types of configuration that apply to Stream forwarder:

inputs.conf: The inputs.conf file inside Splunk_TA_stream/local must contain the location (URI) of your splunk_app_stream installation. The streamfwd binary uses this location to retrieve the specific stream capture configurations (protocols, fields, aggregation types, and so on) that you define in the Configure Streams UI. For more information, see Configure Streams in the Splunk Stream User Manual.

streamfwd.conf:Splunk_TA_stream/local/streamfwd.conf lets you specify system-level data capture parameters for the streamfwd binary. For more information, see Configure streamfwd.conf on this page.

Verify location of splunk_app_stream in inputs.conf

Before you set up stream data capture using the Configure Streams UI, make sure that Splunk_TA_stream/local/inputs.conf is configured to communicate with splunk_app_stream.

Open $SPLUNK_HOME/etc/apps/Splunk_TA_stream/local/inputs.conf.

Confirm that the [streamfwd://streamfwd] stanza contains the correct location (URI) of your splunk_app_stream installation. For example:

Note: The splunk_app_stream URI supports http and https protocols. If you enable SSL for the streamfwd binary, you must change the URI path to specify https. If you change the http port, you must change the URI path to specify the new port.

Configure Stream forwarder identifier

If you are using a deployment server, when you set or modify the stream_forwarder_id of a specific Stream forwarder while a streamfwd process is running, you must restart the universal forwarder for the changes to apply to the stream_forwarder_id.

Note: Multiple Stream forwarder deployments can share the same stream_forwarder_id.

The streamfwd.conf configuration file comes with both Splunk_TA_stream and indpendent Stream forwarder, and is located in $SPLUNK_HOME/etc/apps/Splunk_TA_stream/default/ and /opt/streamfwd/default/, respectively.

Caution: Do not change or copy the streamfwd.conf file located in the the default directory. To modify the streamfwd.conf file, create a new version of streamfwd.conf in the local directory. For more information, see Configuration file directories in the Splunk Enterprise Admin Manual.

Enable loopback capture

Stream forwarder does not capture local loopback traffic by default. To capture loopback traffic, add the appropriate streamfwdcapture parameter to streamfwd.conf, as shown in streamfwdcapture Example 1 on this page.

Streamfwd.conf parameters

The streamfwd.conf configuration file accepts these parameter.

Parameter

Description

Value type

Default value

clientIpSslHashBytes

Defines number of client IP octets to use for SSL processor thread hash algorithm. (min value = 0; max value = 4) Applies only if you have _disabled_ useGlobalSSLSessionKeyCache

Enables sharing of SSL cache across processing threads. Set to true to share.

boolean

false

usePacketMemoryPool

When set to true, Stream forwarder uses a pool allocator to allot memory for storing network packets. Because the pool allocator does not release unused memory back to the operating system, setting this parameter to true may result in high memory usage. Set to true only when Stream forwarder is running on a dedicated capture server that processes large traffic volumes.

boolean

false

Note: For a complete list of streamfwd.conf parameters, see streamfwd.conf.spec in $SPLUNK_HOME/etc/apps/Splunk_TA_stream/README.

Use tcpServer to specify endpoints

Stream forwarder automatically detects the client and server endpoints when it captures the beginnings of TCP connections. If it starts capturing traffic after establishing a TCP connection, Stream forwarder normally assumes that the sender of the first packet it sees is the client.

You can modify this behavior by using the tcpServer parameter to define the endpoints of specific TCP servers. If the sender of a packet matches the endpoint, Stream forwarder correctly categorizes it as a server response packet.

tcpServer examples

Example 1: Single HTTP server endpoint

tcpServer.N.address = 192.168.1.102
tcpServer.N.port = 80

Example 2: Wildcard endpoint

Use sslServer to specify encrypted/decrypted traffic

Stream forwarder detects endpoint encryption, and attempts to decrypt SSL sessions using the available private keys. Optionally, you can explicitly define the traffic as encrypted by adding sslServer parameters.

sslServer.<N>.address = 192.168.1.102
sslServer.<N>.port = 443

Use streamfwdcapture to specify network interfaces

By default streamfwd.conf listens for traffic on all available network interfaces. Using the streamfwdcapture parameter you can restrict data capture to specific interfaces only.

Lets you set a BPF (Berkeley Packet Filter) for kernel-level packet filtering. The value of this tag must comply with BPF syntax. Only one Filter variable per streamfwdcapture parameter is supported.

streamfwdcapture.<N>.repeat

True means to play back the pcap file repeatedly for continuous load

streamfwdcapture.<N>.sysTime

True means to use the system time for packet timestamps instead of actual time timestamps from pcap file

streamfwdcapture.<N>.bitsPerSecond

Rate limiter, defaults to 10 Mbps if undefined and <Repeat> is true

To restrict data capture to specific network interfaces, you must insert a [streamfwd] stanza into streamfwd.conf. You can use streamfwdcatpure parameters to specify multiple network interfaces in a single streamfwd.conf file. For example, to specify two network interfaces - eth0 and eth1 -configured with different BPF filters on *nix:

On Windows, you can substitute the streamfwdcapture.<N>.interface or streamfwdcapture.<N>.InterfaceRegex name (such as \Device\NPF_{D6995D00-B75C-48DB-99AA-69F0150126BC}) with the <Alias> or <Description> value returned by the --iflist command line option.

For example, streamfwdcapture.<N>.interface = Local Area Connection 2 or streamfwdcapture.<N>.InterfaceRegex = Local Area.*.

streamfwdcapture examples

Example 1: Configure streamfwd.conf to include local loopback capture

Stream forwarder by default does not capture traffic that originates and terminates on the same machine. You can enable capture of this "local loopback" traffic using the streamfwdcapture parameter in the configuration file:

streamfwdcapture.<N>.interface = lo0

Note: you cannot use streamfwdcapture.<N>.interfaceRegex> parameter to specify local loopback interfaces.

Example 2: Configure streamfwd.conf for use across multiple systems

You might want to maintain a master copy of streamfwd.conf that you can reuse across multiple systems that have different network device names. The following streamfwd.conf configuration listens on all matching interfaces found (excluding local loopback interfaces).

streamfwdcapture.<N>.interfaceRegex = .*

Note that this configuration may generate startup warnings for any devices that do not support passive data capture.

Example 3: Capture data on specific network interfaces

In this example, on a system with 8 network interfaces, streamfwd.conf would listen only for tcp port 80 traffic on only two of those interfaces (4 and 5):

Enter your email address, and someone from the documentation team will respond to you:

Send me a copy of this feedback

Please provide your comments here. Ask a question or make a suggestion.

Feedback submitted, thanks!

You must be logged into splunk.com in order to post comments.
Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic.
If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk,
consider posting a question to Splunkbase Answers.

0
out of 1000 Characters

Your Comment Has Been Posted Above

We use our own and third-party cookies to provide you with a great online experience. We also use these cookies to improve our products and services, support our marketing campaigns, and advertise to you on our website and other websites. Some cookies may continue to collect information after you have left our website.
Learn more (including how to update your settings) here »