Combine elements of each key in DStream's RDDs using custom function. This is similar to the
combineByKey for RDDs. Please refer to combineByKey in
org.apache.spark.rdd.PairRDDFunctions for more information.

Combine elements of each key in DStream's RDDs using custom function. This is similar to the
combineByKey for RDDs. Please refer to combineByKey in
org.apache.spark.rdd.PairRDDFunctions for more information.

Return a new DStream in which each RDD contains the count of distinct elements in
RDDs in a sliding window over this DStream.

Return a new DStream in which each RDD contains the count of distinct elements in
RDDs in a sliding window over this DStream. Hash partitioning is used to generate the RDDs
with numPartitions partitions.

windowDuration

width of the window; must be a multiple of this DStream's
batching interval

slideDuration

sliding interval of the window (i.e., the interval after which
the new DStream will generate RDDs); must be a multiple of this
DStream's batching interval

Return a new DStream in which each RDD contains the count of distinct elements in
RDDs in a sliding window over this DStream.

Return a new DStream in which each RDD contains the count of distinct elements in
RDDs in a sliding window over this DStream. Hash partitioning is used to generate the RDDs
with Spark's default number of partitions.

windowDuration

width of the window; must be a multiple of this DStream's
batching interval

slideDuration

sliding interval of the window (i.e., the interval after which
the new DStream will generate RDDs); must be a multiple of this
DStream's batching interval

Return a new DStream in which each RDD has a single element generated by counting the number
of elements in a window over this DStream.

Return a new DStream in which each RDD has a single element generated by counting the number
of elements in a window over this DStream. windowDuration and slideDuration are as defined in
the window() operation. This is equivalent to window(windowDuration, slideDuration).count()

Return a new DStream by applying groupByKey on each RDD of this DStream.

Return a new DStream by applying groupByKey on each RDD of this DStream.
Therefore, the values for each key in this DStream's RDDs are grouped into a
single sequence to generate the RDDs of the new DStream. org.apache.spark.Partitioner
is used to control the partitioning of each RDD.

Return a new DStream by applying groupByKey over a sliding window on this DStream.

Return a new DStream by applying groupByKey over a sliding window on this DStream.
Similar to DStream.groupByKey(), but applies it over a sliding window.
Hash partitioning is used to generate the RDDs with numPartitions partitions.

windowDuration

width of the window; must be a multiple of this DStream's
batching interval

slideDuration

sliding interval of the window (i.e., the interval after which
the new DStream will generate RDDs); must be a multiple of this
DStream's batching interval

Return a new DStream by applying groupByKey over a sliding window. Similar to
DStream.groupByKey(), but applies it over a sliding window. Hash partitioning is used to
generate the RDDs with Spark's default number of partitions.

windowDuration

width of the window; must be a multiple of this DStream's
batching interval

slideDuration

sliding interval of the window (i.e., the interval after which
the new DStream will generate RDDs); must be a multiple of this
DStream's batching interval

Return a new DStream by applying groupByKey over a sliding window. This is similar to
DStream.groupByKey() but applies it over a sliding window. The new DStream generates RDDs
with the same interval as this DStream. Hash partitioning is used to generate the RDDs with
Spark's default number of partitions.

windowDuration

width of the window; must be a multiple of this DStream's
batching interval

:: Experimental ::
Return a JavaMapWithStateDStream by applying a function to every key-value element of
this stream, while maintaining some state data for each unique key.

:: Experimental ::
Return a JavaMapWithStateDStream by applying a function to every key-value element of
this stream, while maintaining some state data for each unique key. The mapping function
and other specification (e.g. partitioners, timeouts, initial state data, etc.) of this
transformation can be specified using StateSpec class. The state data is accessible in
as a parameter of type State in the mapping function.

Return a new DStream by applying reduceByKey to each RDD. The values for each key are
merged using the supplied reduce function. org.apache.spark.Partitioner is used to control
the partitioning of each RDD.

Return a new DStream by applying reduceByKey to each RDD. The values for each key are
merged using the supplied reduce function. Hash partitioning is used to generate the RDDs
with numPartitions partitions.

Return a new DStream by applying reduceByKey to each RDD. The values for each key are
merged using the associative and commutative reduce function. Hash partitioning is used to
generate the RDDs with Spark's default number of partitions.

Return a new DStream by applying incremental reduceByKey over a sliding window.

Return a new DStream by applying incremental reduceByKey over a sliding window.
The reduced value of over a new window is calculated using the old window's reduce value :

reduce the new values that entered the window (e.g., adding new counts)
2. "inverse reduce" the old values that left the window (e.g., subtracting old counts)
This is more efficient that reduceByKeyAndWindow without "inverse reduce" function.
However, it is applicable to only "invertible reduce functions".

reduceFunc

associative and commutative reduce function

invReduceFunc

inverse function

windowDuration

width of the window; must be a multiple of this DStream's
batching interval

slideDuration

sliding interval of the window (i.e., the interval after which
the new DStream will generate RDDs); must be a multiple of this
DStream's batching interval

partitioner

Partitioner for controlling the partitioning of each RDD in the new
DStream.

filterFunc

function to filter expired key-value pairs;
only pairs that satisfy the function are retained
set this to null if you do not want to filter

Return a new DStream by applying incremental reduceByKey over a sliding window.

Return a new DStream by applying incremental reduceByKey over a sliding window.
The reduced value of over a new window is calculated using the old window's reduce value :

reduce the new values that entered the window (e.g., adding new counts)
2. "inverse reduce" the old values that left the window (e.g., subtracting old counts)
This is more efficient that reduceByKeyAndWindow without "inverse reduce" function.
However, it is applicable to only "invertible reduce functions".
Hash partitioning is used to generate the RDDs with numPartitions partitions.

reduceFunc

associative and commutative reduce function

invReduceFunc

inverse function

windowDuration

width of the window; must be a multiple of this DStream's
batching interval

slideDuration

sliding interval of the window (i.e., the interval after which
the new DStream will generate RDDs); must be a multiple of this
DStream's batching interval

numPartitions

number of partitions of each RDD in the new DStream.

filterFunc

function to filter expired key-value pairs;
only pairs that satisfy the function are retained
set this to null if you do not want to filter

Return a new DStream by reducing over a using incremental computation.

Return a new DStream by reducing over a using incremental computation.
The reduced value of over a new window is calculated using the old window's reduce value :

reduce the new values that entered the window (e.g., adding new counts)
2. "inverse reduce" the old values that left the window (e.g., subtracting old counts)
This is more efficient that reduceByKeyAndWindow without "inverse reduce" function.
However, it is applicable to only "invertible reduce functions".
Hash partitioning is used to generate the RDDs with Spark's default number of partitions.

Return a new DStream by applying reduceByKey over a sliding window. This is similar to
DStream.reduceByKey() but applies it over a sliding window. Hash partitioning is used to
generate the RDDs with numPartitions partitions.

reduceFunc

associative and commutative reduce function

windowDuration

width of the window; must be a multiple of this DStream's
batching interval

slideDuration

sliding interval of the window (i.e., the interval after which
the new DStream will generate RDDs); must be a multiple of this
DStream's batching interval

Return a new DStream by applying reduceByKey over a sliding window. This is similar to
DStream.reduceByKey() but applies it over a sliding window. Hash partitioning is used to
generate the RDDs with Spark's default number of partitions.

reduceFunc

associative and commutative reduce function

windowDuration

width of the window; must be a multiple of this DStream's
batching interval

slideDuration

sliding interval of the window (i.e., the interval after which
the new DStream will generate RDDs); must be a multiple of this
DStream's batching interval

Create a new DStream by applying reduceByKey over a sliding window on this DStream.

Create a new DStream by applying reduceByKey over a sliding window on this DStream.
Similar to DStream.reduceByKey(), but applies it over a sliding window. The new DStream
generates RDDs with the same interval as this DStream. Hash partitioning is used to generate
the RDDs with Spark's default number of partitions.

reduceFunc

associative and commutative reduce function

windowDuration

width of the window; must be a multiple of this DStream's
batching interval

Return a new DStream in which each RDD has a single element generated by reducing all
elements in a sliding window over this DStream.

Return a new DStream in which each RDD has a single element generated by reducing all
elements in a sliding window over this DStream. However, the reduction is done incrementally
using the old window's reduced value :

reduce the new values that entered the window (e.g., adding new counts)
2. "inverse reduce" the old values that left the window (e.g., subtracting old counts)
This is more efficient than reduceByWindow without "inverse reduce" function.
However, it is applicable to only "invertible reduce functions".

Return a new "state" DStream where the state for each key is updated by applying
the given function on the previous state of the key and the new values of the key.

Return a new "state" DStream where the state for each key is updated by applying
the given function on the previous state of the key and the new values of the key.
org.apache.spark.Partitioner is used to control the partitioning of each RDD.

S

State type

updateFunc

State update function. If this function returns None, then
corresponding state key-value pair will be eliminated.

partitioner

Partitioner for controlling the partitioning of each RDD in the new
DStream.

Return a new "state" DStream where the state for each key is updated by applying
the given function on the previous state of the key and the new values of the key.

Return a new "state" DStream where the state for each key is updated by applying
the given function on the previous state of the key and the new values of the key.
org.apache.spark.Partitioner is used to control the partitioning of each RDD.

S

State type

updateFunc

State update function. If this function returns None, then
corresponding state key-value pair will be eliminated.

partitioner

Partitioner for controlling the partitioning of each RDD in the new
DStream.

Return a new "state" DStream where the state for each key is updated by applying
the given function on the previous state of the key and the new values of each key.

Return a new "state" DStream where the state for each key is updated by applying
the given function on the previous state of the key and the new values of each key.
Hash partitioning is used to generate the RDDs with numPartitions partitions.

S

State type

updateFunc

State update function. If this function returns None, then
corresponding state key-value pair will be eliminated.

Return a new "state" DStream where the state for each key is updated by applying
the given function on the previous state of the key and the new values of each key.

Return a new "state" DStream where the state for each key is updated by applying
the given function on the previous state of the key and the new values of each key.
Hash partitioning is used to generate the RDDs with Spark's default number of partitions.

S

State type

updateFunc

State update function. If this function returns None, then
corresponding state key-value pair will be eliminated.