Mainflux.mainflux/consumers/writers/postgres
b1ackd0t 4401e79a0d
NOISSUE - Add Subscriber Config (#1896)
* Replace Nats with Nats Jestream For PubSub

Signed-off-by: rodneyosodo <blackd0t@protonmail.com>

* Add Stream Description

Signed-off-by: rodneyosodo <blackd0t@protonmail.com>

* Fix connection leak in NATS publisher

The publisher struct in pkg/messaging/nats/publisher.go was modified to include a new `conn` field of type `*broker.Conn`. This change was made to fix a connection leak issue in the NATS publisher.

The `NewPublisher` function was updated to assign the `conn` parameter to the new `conn` field in the publisher struct.

Additionally, the `Close` method in the publisher struct was modified to close the `conn` connection.

This commit fixes the connection leak issue in the NATS publisher and ensures that connections are properly closed.

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>

* Setup subscriber config to contain handler topic and ID

Signed-off-by: rodneyosodo <blackd0t@protonmail.com>

* Add delivery policy

Signed-off-by: rodneyosodo <blackd0t@protonmail.com>

* Avoid duplicate messages

Signed-off-by: rodneyosodo <blackd0t@protonmail.com>

* Rename to DeliveryPolicy

Signed-off-by: rodneyosodo <blackd0t@protonmail.com>

* Fix tests

Signed-off-by: rodneyosodo <blackd0t@protonmail.com>

* Not check for data result set when we are returning subset of messages

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com>

* For unsubscribe remove config

Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com>

* Fix comment

Signed-off-by: rodneyosodo <blackd0t@protonmail.com>

---------

Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com>
2023-10-23 15:27:15 +02:00
..
README.md NOISSUE - Make MQTT Broker Configurable (#1904) 2023-10-19 21:01:09 +02:00
consumer.go NOISSUE - Add More Linters (#1924) 2023-10-16 11:43:33 +02:00
consumer_test.go NOISSUE - Change import name aliases (#1868) 2023-08-11 11:30:25 +02:00
doc.go NOISSUE - Switch to Consumers interface (#1316) 2021-01-11 23:55:34 +01:00
init.go NOISSUE - Add Subscriber Config (#1896) 2023-10-23 15:27:15 +02:00
setup_test.go NOISSUE - Change import name aliases (#1868) 2023-08-11 11:30:25 +02:00

README.md

Postgres writer

Postgres writer provides message repository implementation for Postgres.

Configuration

The service is configured using the environment variables presented in the following table. Note that any unset variables will be replaced with their default values.

Variable Description Default
MF_POSTGRES_WRITER_LOG_LEVEL Service log level info
MF_POSTGRES_WRITER_CONFIG_PATH Config file path with Message broker subjects list, payload type and content-type /config.toml
MF_POSTGRES_WRITER_HTTP_HOST Service HTTP host localhost
MF_POSTGRES_WRITER_HTTP_PORT Service HTTP port 9010
MF_POSTGRES_WRITER_HTTP_SERVER_CERT Service HTTP server certificate path ""
MF_POSTGRES_WRITER_HTTP_SERVER_KEY Service HTTP server key ""
MF_POSTGRES_HOST Postgres DB host postgres
MF_POSTGRES_PORT Postgres DB port 5432
MF_POSTGRES_USER Postgres user mainflux
MF_POSTGRES_PASS Postgres password mainflux
MF_POSTGRES_NAME Postgres database name messages
MF_POSTGRES_SSL_MODE Postgres SSL mode disabled
MF_POSTGRES_SSL_CERT Postgres SSL certificate path ""
MF_POSTGRES_SSL_KEY Postgres SSL key ""
MF_POSTGRES_SSL_ROOT_CERT Postgres SSL root certificate path ""
MF_MESSAGE_BROKER_URL Message broker instance URL nats://localhost:4222
MF_JAEGER_URL Jaeger server URL http://jaeger:14268/api/traces
MF_SEND_TELEMETRY Send telemetry to mainflux call home server true
MF_POSTGRES_WRITER_INSTANCE_ID Service instance ID ""

Deployment

The service itself is distributed as Docker container. Check the postgres-writer service section in docker-compose to see how service is deployed.

To start the service, execute the following shell script:

# download the latest version of the service
git clone https://github.com/mainflux/mainflux

cd mainflux

# compile the postgres writer
make postgres-writer

# copy binary to bin
make install

# Set the environment variables and run the service
MF_POSTGRES_WRITER_LOG_LEVEL=[Service log level] \
MF_POSTGRES_WRITER_CONFIG_PATH=[Config file path with Message broker subjects list, payload type and content-type] \
MF_POSTGRES_WRITER_HTTP_HOST=[Service HTTP host] \
MF_POSTGRES_WRITER_HTTP_PORT=[Service HTTP port] \
MF_POSTGRES_WRITER_HTTP_SERVER_CERT=[Service HTTP server cert] \
MF_POSTGRES_WRITER_HTTP_SERVER_KEY=[Service HTTP server key] \
MF_POSTGRES_HOST=[Postgres host] \
MF_POSTGRES_PORT=[Postgres port] \
MF_POSTGRES_USER=[Postgres user] \
MF_POSTGRES_PASS=[Postgres password] \
MF_POSTGRES_NAME=[Postgres database name] \
MF_POSTGRES_SSL_MODE=[Postgres SSL mode] \
MF_POSTGRES_SSL_CERT=[Postgres SSL cert] \
MF_POSTGRES_SSL_KEY=[Postgres SSL key] \
MF_POSTGRES_SSL_ROOT_CERT=[Postgres SSL Root cert] \
MF_MESSAGE_BROKER_URL=[Message broker instance URL] \
MF_JAEGER_URL=[Jaeger server URL] \
MF_SEND_TELEMETRY=[Send telemetry to mainflux call home server] \
MF_POSTGRES_WRITER_INSTANCE_ID=[Service instance ID] \

$GOBIN/mainflux-postgres-writer

Usage

Starting service will start consuming normalized messages in SenML format.