History

b1ackd0t 4401e79a0d NOISSUE - Add Subscriber Config (#1896 ) * Replace Nats with Nats Jestream For PubSub Signed-off-by: rodneyosodo <blackd0t@protonmail.com> * Add Stream Description Signed-off-by: rodneyosodo <blackd0t@protonmail.com> * Fix connection leak in NATS publisher The publisher struct in pkg/messaging/nats/publisher.go was modified to include a new `conn` field of type `broker.Conn`. This change was made to fix a connection leak issue in the NATS publisher. The `NewPublisher` function was updated to assign the `conn` parameter to the new `conn` field in the publisher struct. Additionally, the `Close` method in the publisher struct was modified to close the `conn` connection. This commit fixes the connection leak issue in the NATS publisher and ensures that connections are properly closed. Signed-off-by: Rodney Osodo <socials@rodneyosodo.com> Setup subscriber config to contain handler topic and ID Signed-off-by: rodneyosodo <blackd0t@protonmail.com> * Add delivery policy Signed-off-by: rodneyosodo <blackd0t@protonmail.com> * Avoid duplicate messages Signed-off-by: rodneyosodo <blackd0t@protonmail.com> * Rename to DeliveryPolicy Signed-off-by: rodneyosodo <blackd0t@protonmail.com> * Fix tests Signed-off-by: rodneyosodo <blackd0t@protonmail.com> * Not check for data result set when we are returning subset of messages Signed-off-by: Rodney Osodo <socials@rodneyosodo.com> Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com> * For unsubscribe remove config Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com> * Fix comment Signed-off-by: rodneyosodo <blackd0t@protonmail.com> --------- Signed-off-by: rodneyosodo <blackd0t@protonmail.com> Signed-off-by: Rodney Osodo <socials@rodneyosodo.com> Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com>		2023-10-23 15:27:15 +02:00
..
README.md	NOISSUE - Make MQTT Broker Configurable (#1904 )	2023-10-19 21:01:09 +02:00
consumer.go	NOISSUE - Trace consume operations (#1778 )	2023-07-22 14:38:16 +02:00
consumer_test.go	NOISSUE - Change import name aliases (#1868 )	2023-08-11 11:30:25 +02:00
doc.go	NOISSUE - Switch to Consumers interface (#1316 )	2021-01-11 23:55:34 +01:00
init.go	NOISSUE - Add Subscriber Config (#1896 )	2023-10-23 15:27:15 +02:00
setup_test.go	NOISSUE - Change import name aliases (#1868 )	2023-08-11 11:30:25 +02:00

README.md

Cassandra writer

Cassandra writer provides message repository implementation for Cassandra.

Configuration

The service is configured using the environment variables presented in the following table. Note that any unset variables will be replaced with their default values.

Variable	Description	Default
MF_CASSANDRA_WRITER_LOG_LEVEL	Log level for Cassandra writer (debug, info, warn, error)	info
MF_CASSANDRA_WRITER_CONFIG_PATH	Config file path with NATS subjects list, payload type and content-type	/config.toml
MF_CASSANDRA_WRITER_HTTP_HOST	Cassandra service HTTP host
MF_CASSANDRA_WRITER_HTTP_PORT	Cassandra service HTTP port	9004
MF_CASSANDRA_WRITER_HTTP_SERVER_CERT	Cassandra service HTTP server certificate path
MF_CASSANDRA_WRITER_HTTP_SERVER_KEY	Cassandra service HTTP server key path
MF_CASSANDRA_CLUSTER	Cassandra cluster comma separated addresses	127.0.0.1
MF_CASSANDRA_KEYSPACE	Cassandra keyspace name	mainflux
MF_CASSANDRA_USER	Cassandra DB username	mainflux
MF_CASSANDRA_PASS	Cassandra DB password	mainflux
MF_CASSANDRA_PORT	Cassandra DB port	9042
MF_MESSAGE_BROKER_URL	Message broker instance URL	nats://localhost:4222
MF_JAEGER_URL	Jaeger server URL	http://jaeger:14268/api/traces
MF_SEND_TELEMETRY	Send telemetry to mainflux call home server	true
MF_CASSANDRA_WRITER_INSANCE_ID	Cassandra writer instance ID

Deployment

The service itself is distributed as Docker container. Check the cassandra-writer service section in docker-compose to see how service is deployed.

To start the service, execute the following shell script:

# download the latest version of the service
git clone https://github.com/mainflux/mainflux

cd mainflux

# compile the cassandra writer
make cassandra-writer

# copy binary to bin
make install

# Set the environment variables and run the service
MF_CASSANDRA_WRITER_LOG_LEVEL=[Cassandra writer log level] \
MF_CASSANDRA_WRITER_CONFIG_PATH=[Config file path with NATS subjects list, payload type and content-type] \
MF_CASSANDRA_WRITER_HTTP_HOST=[Cassandra service HTTP host] \
MF_CASSANDRA_WRITER_HTTP_PORT=[Cassandra service HTTP port] \
MF_CASSANDRA_WRITER_HTTP_SERVER_CERT=[Cassandra service HTTP server cert] \
MF_CASSANDRA_WRITER_HTTP_SERVER_KEY=[Cassandra service HTTP server key] \
MF_CASSANDRA_CLUSTER=[Cassandra cluster comma separated addresses] \
MF_CASSANDRA_KEYSPACE=[Cassandra keyspace name] \
MF_CASSANDRA_USER=[Cassandra DB username] \
MF_CASSANDRA_PASS=[Cassandra DB password] \
MF_CASSANDRA_PORT=[Cassandra DB port] \
MF_MESSAGE_BROKER_URL=[Message Broker instance URL] \
MF_JAEGER_URL=[Jaeger server URL] \
MF_SEND_TELEMETRY=[Send telemetry to mainflux call home server] \
MF_CASSANDRA_WRITER_INSANCE_ID=[Cassandra writer instance ID] \
$GOBIN/mainflux-cassandra-writer

Using docker-compose

This service can be deployed using docker containers. Docker compose file is available in <project_root>/docker/addons/cassandra-writer/docker-compose.yml. In order to run all Mainflux core services, as well as mentioned optional ones, execute following command:

./docker/addons/cassandra-writer/init.sh

Usage

Starting service will start consuming normalized messages in SenML format.