Motivation

Why I built riko

Wanting to create custom pipes, I came across pipe2py which translated a
Yahoo! Pipe into python code. pipe2py suited my needs at the time
but was unmaintained and lacked asynchronous or parallel processing.

riko addresses the shortcomings of pipe2py but removed support for
importing Yahoo! Pipes json workflows. riko contains ~ 40 built-in
modules, aka pipes, that allow you to programatically perform most of the
tasks Yahoo! Pipes allowed.

Why you should use riko

riko provides a number of benefits / differences from other stream processing
applications such as Huginn, Flink, Spark, and Storm [6]. Namely:

Installation

Design Principles

The primary data structures in riko are the item and stream. An item
is just a python dictionary, and a stream is an iterator of items. You can
create a stream manually with something as simple as
[{'content': 'hello world'}]. You manipulate streams in
riko via pipes. A pipe is simply a function that accepts either a
stream or item, and returns a stream. pipes are composable: you
can use the output of one pipe as the input to another pipe.

rikopipes come in two flavors; operators and processors.
operators operate on an entire stream at once and are unable to handle
individual items. Example operators include count, pipefilter,
and reverse.

operators are split into sub-types of aggregators
and composers. aggregators, e.g., count, combine
all items of an input stream into a new stream with a single item;
while composers, e.g., filter, create a new stream containing
some or all items of an input stream.

CLI Examples

Now to execute flow.py, type the command runpipe flow. You should
then see the following output in your terminal:

https://google.co.uk

runpipe will also search the examples directory for workflows. Type
runpipe demo and you should see the following output:

Deadline to clear up health law eligibility near 682

Scripts

riko comes with a built in task manager manage.

Setup

pip install riko[develop]

Examples

Run python linter and nose tests

manage lint
manage test

Contributing

Please mimic the coding style/conventions used in this repo.
If you add new classes or functions, please add the appropriate doc blocks with
examples. Also, make sure the python linter and nose tests pass.