Publish & Subscribe

Publisher

Complete source code: github.com/ThreeDotsLabs/watermill/message/pubsub.go

// ...
type Publisher interface {
	// Publish publishes the provided messages to the given topic.
	//
	// Publish can be synchronous or asynchronous - it depends on the implementation.
	//
	// Most publisher implementations do not support atomic message publishing.
	// This means that if publishing one of the messages fails, the next one will not be published.
	//
	// Publish must be thread-safe.
	Publish(topic string, messages ...*Message) error
	// If the publisher is asynchronous, Close should flush unsent messages.
	Close() error
}
// ...

Publishing Multiple Messages

Most publisher implementations do not support atomic message publishing. This means that if publishing one of the messages fails, the next one will not be published.

Asynchronous Publishing

Publishing can be synchronous or asynchronous - it depends on the implementation.

Close()

If the publisher is asynchronous, Close should flush unsent messages. Don't forget to close the subscribers. Otherwise, you might lose some messages.

Subscriber

Complete source code: github.com/ThreeDotsLabs/watermill/message/pubsub.go

// ...
type Subscriber interface {
	// Subscribe returns an output channel with messages from the provided topic.
	// The channel will close when Close() on the subscriber is called.
	//
	// To receive the next message, `Ack()` must be called on the received message.
	// If the message processing fails and the message should be redelivered, `Nack()` should be called.
	//
	// When the provided ctx is canceled, the subscriber will close the subscription and close the output channel.
	// The provided ctx is set to all generated messages.
	// When Ack or Nack is called on the message, the message's context will be canceled.
	Subscribe(ctx context.Context, topic string) (
}

Ack/Nack Mechanism

Subscribers are responsible for handling Ack and Nack from messages. Proper implementation should wait for Ack or Nack before consuming the next message.

Important subscriber implementation tip: It's crucial to send Ack/offset to the message's storage/agent after Ack from Watermill message. Otherwise, if the process dies before processing the message, there's a possibility of losing the message.

Close()

Close will close all subscriptions and their output channels, and flush offsets if necessary.

At Least Once Delivery

Watermill is built using at least once delivery semantics. This means that if an error occurs when processing a message and it's not possible to send Ack, the message will be redelivered.

You need to keep this in mind and build your application for idempotent processing or implement a retry mechanism.

Unfortunately, creating a generic retry middleware is not possible, so we encourage you to build your own implementation.

Generic Testing

Every Pub/Sub is similar in most aspects. To avoid writing separate tests for each Pub/Sub implementation, we have created a test suite that any Pub/Sub should pass.

These tests can be found in pubsub/tests/test_pubsub.go.

Built-in Implementation

To check the available Pub/Sub implementations, please refer to the supported Pub/Sub.

Implementing Custom Pub/Sub

For instructions on introducing support for a new Pub/Sub, please refer to "Implementing Custom Pub/Sub".