Mainflux.mainflux/consumers/writers/mongodb
b1ackd0t 4401e79a0d
NOISSUE - Add Subscriber Config (#1896)
* Replace Nats with Nats Jestream For PubSub

Signed-off-by: rodneyosodo <blackd0t@protonmail.com>

* Add Stream Description

Signed-off-by: rodneyosodo <blackd0t@protonmail.com>

* Fix connection leak in NATS publisher

The publisher struct in pkg/messaging/nats/publisher.go was modified to include a new `conn` field of type `*broker.Conn`. This change was made to fix a connection leak issue in the NATS publisher.

The `NewPublisher` function was updated to assign the `conn` parameter to the new `conn` field in the publisher struct.

Additionally, the `Close` method in the publisher struct was modified to close the `conn` connection.

This commit fixes the connection leak issue in the NATS publisher and ensures that connections are properly closed.

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>

* Setup subscriber config to contain handler topic and ID

Signed-off-by: rodneyosodo <blackd0t@protonmail.com>

* Add delivery policy

Signed-off-by: rodneyosodo <blackd0t@protonmail.com>

* Avoid duplicate messages

Signed-off-by: rodneyosodo <blackd0t@protonmail.com>

* Rename to DeliveryPolicy

Signed-off-by: rodneyosodo <blackd0t@protonmail.com>

* Fix tests

Signed-off-by: rodneyosodo <blackd0t@protonmail.com>

* Not check for data result set when we are returning subset of messages

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com>

* For unsubscribe remove config

Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com>

* Fix comment

Signed-off-by: rodneyosodo <blackd0t@protonmail.com>

---------

Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com>
2023-10-23 15:27:15 +02:00
..
README.md NOISSUE - Make MQTT Broker Configurable (#1904) 2023-10-19 21:01:09 +02:00
consumer.go NOISSUE - Add Subscriber Config (#1896) 2023-10-23 15:27:15 +02:00
consumer_test.go NOISSUE - Change import name aliases (#1868) 2023-08-11 11:30:25 +02:00
doc.go NOISSUE - Switch to Consumers interface (#1316) 2021-01-11 23:55:34 +01:00
setup_test.go NOISSUE - Change import name aliases (#1868) 2023-08-11 11:30:25 +02:00

README.md

MongoDB writer

MongoDB writer provides message repository implementation for MongoDB.

Configuration

The service is configured using the environment variables presented in the following table. Note that any unset variables will be replaced with their default values.

Variable Description Default
MF_MONGO_WRITER_LOG_LEVEL Log level for MongoDB writer info
MF_MONGO_WRITER_CONFIG_PATH Config file path with Message broker subjects list, payload type and content-type /config.toml
MF_MONGO_WRITER_HTTP_HOST Service HTTP host localhost
MF_MONGO_WRITER_HTTP_PORT Service HTTP port 9010
MF_MONGO_WRITER_HTTP_SERVER_CERT Service HTTP server certificate path ""
MF_MONGO_WRITER_HTTP_SERVER_KEY Service HTTP server key ""
MF_MONGO_NAME Default MongoDB database name messages
MF_MONGO_HOST Default MongoDB database host localhost
MF_MONGO_PORT Default MongoDB database port 27017
MF_MESSAGE_BROKER_URL Message broker instance URL nats://localhost:4222
MF_JAEGER_URL Jaeger server URL http://jaeger:14268/api/traces
MF_SEND_TELEMETRY Send telemetry to mainflux call home server true
MF_MONGO_WRITER_INSTANCE_ID MongoDB writer instance ID ""

Deployment

The service itself is distributed as Docker container. Check the mongodb-writer service section in docker-compose to see how service is deployed.

To start the service, execute the following shell script:

# download the latest version of the service
git clone https://github.com/mainflux/mainflux

cd mainflux

# compile the mongodb writer
make mongodb-writer

# copy binary to bin
make install

# Set the environment variables and run the service
MF_MONGO_WRITER_LOG_LEVEL=[MongoDB writer log level] \
MF_MONGO_WRITER_CONFIG_PATH=[Configuration file path with Message broker subjects list] \
MF_MONGO_WRITER_HTTP_HOST=[Service HTTP host] \
MF_MONGO_WRITER_HTTP_PORT=[Service HTTP port] \
MF_MONGO_WRITER_HTTP_SERVER_CERT=[Service HTTP server certificate] \
MF_MONGO_WRITER_HTTP_SERVER_KEY=[Service HTTP server key] \
MF_MONGO_NAME=[MongoDB database name] \
MF_MONGO_HOST=[MongoDB database host] \
MF_MONGO_PORT=[MongoDB database port] \
MF_MESSAGE_BROKER_URL=[Message broker instance URL] \
MF_JAEGER_URL=[Jaeger server URL] \
MF_SEND_TELEMETRY=[Send telemetry to mainflux call home server] \
MF_MONGO_WRITER_INSTANCE_ID=[MongoDB writer instance ID] \

$GOBIN/mainflux-mongodb-writer

Usage

Starting service will start consuming normalized messages in SenML format.