npm

nsqjs

nsqjs

The official NodeJS client for the nsq client protocol. This implementation attempts to be
fully compliant and maintain feature parity with the official Go (go-nsq) and Python (pynsq) clients.

Usage

new Reader(topic, channel, options)

The topic and channel arguments are strings and must be specified. The options
argument is optional. Below are the parameters that can be specified in the
options object.

maxInFlight: 1
The maximum number of messages to process at once. This value is shared between nsqd connections. It's highly recommended that this value is greater than the number of nsqd connections.

heartbeatInterval: 30
The frequency in seconds at which the nsqd will send heartbeats to this Reader.

maxBackoffDuration: 128
The maximum amount of time (seconds) the Reader will backoff for any single backoff
event.

maxAttempts: 0
The number of times a given message will be attempted (given to MESSAGE handler) before it will be handed to the DISCARD handler and then automatically finished. 0 means that there is no limit. If no DISCARD handler is specified and maxAttempts > 0, then the message will be finished automatically when the number of attempts has been exhausted.

requeueDelay: 90,000 (90secs)
The default amount of time (milliseconds) a message requeued should be delayed by before being dispatched by nsqd.

nsqdTCPAddresses
A string or an array of strings representing the host/port pair for nsqd instances.
For example: ['localhost:4150']

lookupdHTTPAddresses
A string or an array of strings representing the host/port pair of nsqlookupd instaces or the full HTTP/HTTPS URIs of the nsqlookupd instances.
For example: ['localhost:4161'], ['http://localhost/lookup'], ['http://localhost/path/lookup?extra_param=true']

lookupdPollInterval: 60
The frequency in seconds for querying lookupd instances.

lookupdPollJitter: 0.3
The jitter applied to the start of querying lookupd instances periodically.

lowRdyTimeout: 50
The timeout in milliseconds for switching between connections when the Reader
maxInFlight is less than the number of connected NSQDs.

tls: false
Use TLS if nsqd has TLS support enabled.

tlsVerification: true
Require verification of the TLS cert. This needs to be false if you're using
a self signed cert.

deflate: false
Use zlib Deflate compression.

deflateLevel: 6
Use zlib Deflate compression level.

snappy: false
Use Snappy compression.

authSecret: null
Authenticate using the provided auth secret.

outputBufferSize: null
The size in bytes of the buffer nsqd will use when writing to this client. -1
disables buffering. outputBufferSize >= 64

outputBufferTimeout: null
The timeout after which any data that nsqd has buffered will be flushed to this client. Value is in milliseconds. outputBufferTimeout >= 1. A value of -1 disables timeouts.

messageTimeout: null
Sets the server-side message timeout in milliseconds for messages delivered to this client.

sampleRate: null
Deliver a percentage of all messages received to this connection. 1 <= sampleRate <= 99

clientId: null
An identifier used to disambiguate this client.

idleTimeout: 0
Socket timeout after idling for the duration in second (default to 0 means disabled).

Reader events are:

Reader.READY or ready

Reader.NOT_READY or not_ready

Reader.MESSAGE or message

Reader.DISCARD or discard

Reader.ERROR or error

Reader.NSQD_CONNECTED or nsqd_connected

Reader.NSQD_CLOSED or nsqd_closed

Reader.MESSAGE and Reader.DISCARD both produce Message objects.
Reader.NSQD_CONNECTED and Reader.NSQD_CLOSED events both provide the host
and port of the nsqd to which the event pertains.

These methods are available on a Reader object:

connect()
Connect to the nsqds specified or connect to nsqds discovered via
lookupd.

close()
Disconnect from all nsqds. Does not wait for in-flight messages to complete.

pause()
Pause the Reader by stopping message flow. Does not affect in-flight
messages.

unpause()
Unpauses the Reader by resuming normal message flow.

isPaused()true if paused, false otherwise.

Message

The following properties and methods are available on Message objects produced by a Reader
instance.

timestamp
Numeric timestamp for the Message provided by nsqd.

attempts
The number of attempts that have been made to process this message.

id
The opaque string id for the Message provided by nsqd.

hasResponded
Boolean for whether or not a response has been sent.

body
The message payload as a Buffer object.

json()
Parses message payload as JSON and caches the result.

timeUntilTimeout(hard=false):
Returns the amount of time until the message times out. If the hard argument
is provided, then it calculates the time until the hard timeout when nsqd
will requeue inspite of touch events.

finish()
Finish the message as successful.

requeue(delay=null, backoff=true)
The delay is in milliseconds. This is how long nsqd will hold on the message
before attempting it again. The backoff parameter indicates that we should
treat this as an error within this process and we need to backoff to recover.

touch()
Tell nsqd that you want extra time to process the message. It extends the
soft timeout by the normal timeout amount.

new Writer(nsqdHost, nsqdPort, options)

Allows messages to be sent to an nsqd.

Available Writer options:

tls: false
Use TLS if nsqd has TLS support enabled.

tlsVerification: true
Require verification of the TLS cert. This needs to be false if you're using
a self signed cert.

deflate: false
Use zlib Deflate compression.

deflateLevel: 6
Use zlib Deflate compression level.

snappy: false
Use Snappy compression.

clientId: null
An identifier used to disambiguate this client.

Writer events are:

Writer.READY or ready

Writer.CLOSED or closed

Writer.ERROR or error

These methods are available on a Writer object:

connect()
Connect to the nsqd specified.

close()
Disconnect from nsqd.

publish(topic, msgs, [callback])topic is a string. msgs is either a string, a Buffer, JSON serializable
object, a list of strings / Buffers / JSON serializable objects. callback takes a single error argument.

deferPublish(topic, msg, timeMs, [callback])topic is a string. msg is either a string, a Buffer, JSON serializable object. timeMs is the delay by which the message should be delivered. callback takes a single error argument.

Example with message timeouts

This script simulates a message that takes a long time to process or at least
longer than the default message timeout. To ensure that the message doesn't
timeout while being processed, touch events are sent to keep it alive.

Fix Reader.close to cleanup all intervals to allow node to exit cleanly.

Upraded Sinon

Removed .eslintrc

0.9.1

Fixed Reader close exceptions. (Thanks @ekristen)

Bump Sinon version

Bump bignumber.js version

0.9

Breaking change: Node versions 6 and greater supported from now on.

Support for deferred message publishing! (thanks @spruce)

Added idleTimeout for Reader and Writer (thanks @abbshr)

Snappy support is now optional. (thanks @bcoe)

Snappy support is now fixed and working with 1.0.0-compat!

Fixed backoff behavior if mulitiple messages fail at the same time. Should recover much faster.

Use TCP NO_DELAY for nsqd connections.

Chores:

Replaced underscore with lodash

Move src to lib

Dropped Babel support

Use Standard JS style

Less flakey tests

0.8.4

Move to ES6 using Babel.

0.7.12

Bug: Fix issue introduced by not sending RDY count to main max-in-flight.
Readers connected to mulitple nsqds do not set RDY count for connections
made after the first nsqd connection.

0.7.11

Improvement: Keep track of touch count in Message instances.

Improvement: Stop sending RDY count to main max-in-flight for newer
versions of nsqd.

Improvement: Make the connect debug message more accurate in Reader.
Previously lookupd poll results suggested new connections were being made
when they were not.

Bug: Non-fatal nsqd errors would cause RDY count to decrease and never
return to normal. This will happen for example when finishing messages
that have exceeded their amount of time to process a message.

0.7.10

Properly handles non-string errors

0.7.9

Treat non-fatal errors appropriately

0.7.7

Build with Node v4

0.7.6

Fix npm install by adding .npmignore.

0.7.3

Slightly better invalid topic and channel error messages.

Handle more conditions for failing to publish a message.

0.7.2

Fix build for iojs and node v0.12

Bumped snappystream version.

0.7.1

Fix connection returning to max connection RDY after backoff

Fix backoff ignored when message.finish is called after backoff event.

0.7.0

Fixes for configuration breakages

Fix for AUTH

Fix for pause / unpause

Automatically finish messages when maxAttempts have been exceeded.

maxAttempts is now by default 0. [ Breaking Change! ]

discarded messages will now be sent to the MESSAGE listener if there's no
DISCARD listener.

Support for emphemeral topics.

Support for 64 char topic / channel names.

Support for Lookupd URLs

Deprecate StateChangeLogger infavor of debug [ Breaking Change! ]

0.6.0

Added support for authentication

Added support for sample rate

Added support for specifying outputBufferSize and outputBufferTimeout

Added support for specifying msg_timeout

Refactored configuration checks

Breaking change for NSQDConnection constructor. NSQDConnection takes an
options argument instead of each option as a parameter.

0.5.1

Fix for not failing when the nsqd isn't available on start.

0.5.0

Reworked FrameBuffer

Added TLS support for Reader and Writer

Added Deflate support

Added Snappy support

0.4.1

Fixed a logging issue on NSQConnection disconnected

0.4.0

Added close, pause, and unpause to Reader

Added callback for Writer publish

Expose error events on Reader/Writer

Expose nsqd connect / disconnect events

Fix crash when Message finish, requeue, etc after nsqd disconnect

Fix lookupd only queried on startup.

0.3.1

Fixed sending an array of Buffers

Fixed sending a message with multi-byte characters

0.3.0

Added Writer implementation

0.2.1

Added prepublish compilation to JavaScript.

0.2.0

ReaderRdy, ConnectionRdy implementation

Reader implementation

Initial documentation

NSQDConnection

Moved defaults to Reader

Support protocol / state logging

connect() now happens on next tick so that it can be called before event
handlers are registered.