Before I go creating a fix I thought it best to summarize and confirm my
expectations on this functionality. Messages from a REGEX_SELECT that don't
match a pattern, and therefore don't get a value assigned to their output
topic value, should be dropped and acked.
*Example:*
{
"parserClassName": "org.apache.metron.parsers.GrokParser",
"sensorTopic": "myInTopic",
...
"parserConfig": {
...,
"kafka.topicField": "output_topic"
},
"fieldTransformations": [
{
"input": [
"message"
],
"output": [
"output_topic"
],
"transformation": "REGEX_SELECT",
"config": {
"world": "^Hello "
}
},
...
}
*Input Records:*
"...sshd[32469]: Hello..."
"...sshd[30432]: Bye..."
*Output:*
Kafka topic = "world" (as determined by the REGEX_SELECT pattern match that
sets the "output_topic" property used by kafka.topicField)
1 record present
contents of that record = our record with "Hello" in it
1 record is dropped ("Bye" record) and will not be forwarded any further
through the pipeline.