Note that arguments to the main method can be passed by providing them in the
heron submit command.

Topology-wide configuration

If you’re building a Python topology using a TopologyBuilder, you can specify configuration for the topology using the set_config method. A topology’s config is a dict in which the keys are a series constants from the api_constants module and values are configuration values for those parameters.

Launching the topology

If you want to submit Python topologies to a Heron cluster, they need to be packaged as a PEX file. In order to produce PEX files, we recommend using a build tool like Pants or Bazel.

If you defined your topology by subclassing the TopologyBuilder class and built a word_count.pex file for that topology in the ~/topology folder. You can submit the topology to a cluster called local like this:

All you need to do is place HeronComponentSpecs as the class attributes
of your topology class, which are returned by the spec() method of
your spout or bolt class. You do not need to run a build method or anything like that; the Topology class will automatically detect which spouts and bolts are included in the topology.

If you use this method to define a new Python topology, you do not need to have a main function.

For bolts, the spec method for spouts takes three optional arguments::

Argument

Data type

Description

Default

name

str

The unique identifier assigned to this bolt or None if you want to use the variable name of the return HeronComponentSpec as the unique identifier for this bolt

Launching

If you defined your topology by subclassing the Topology class,
your main Python file should not contain a main method. You will, however, need to instruct Heron which class contains your topology definition.

Let’s say that you’ve defined a topology by subclassing Topology and built a PEX stored in ~/topology/dist/word_count.pex. The class containing your topology definition is topology.word_count.WordCount. You can submit the topology to a cluster called local like this:

Topology-wide configuration

If you’re building a Python topology by subclassing Topology, you can specify configuration for the topology using the set_config method. A topology’s config is a dict in which the keys are a series constants from the api_constants module and values are configuration values for those parameters.

Declaring output fields using the spec() method

In Python topologies, the output fields of your spouts and bolts
need to be declared by placing outputs class attributes, as there is
no declareOutputFields() method. heronpy enables you to dynamically declare output fields as a list using the
optional_outputs argument in the spec() method.