43a263d7da
* Refactor message broker implementation This commit refactors the Nats message broker implementation to include pubsub options. These changes include: - Adding `Option` func that takes in the URL and prefix - Implement `WithStream` option which can create a different stream for nats stream - Implement `WithExchange` option which can create a different exchaange for rabbitmq channel - Implement `WithPrefix` option which allows to you change the publisher prefix These changes improve the organization and readability of the codebase. Signed-off-by: Rodney Osodo <socials@rodneyosodo.com> Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com> Signed-off-by: rodneyosodo <blackd0t@protonmail.com> * Use redis URL to configure username, password and db Signed-off-by: Rodney Osodo <socials@rodneyosodo.com> Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com> Signed-off-by: rodneyosodo <blackd0t@protonmail.com> * Make event store configurable on dev deployment Signed-off-by: Rodney Osodo <socials@rodneyosodo.com> Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com> Signed-off-by: rodneyosodo <blackd0t@protonmail.com> * Fix adds options to messaging `PubSub` interface Adding options to PubSub interface allows the use of messaging package to do es. The changes in this commit ensure that the code handles errors properly and provides more informative error messages when encountering unexpected types. Signed-off-by: Rodney Osodo <socials@rodneyosodo.com> Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com> Signed-off-by: rodneyosodo <blackd0t@protonmail.com> * Add NATS event publisher implementation This commit adds the implementation of the NATS event publisher. The NATS event publisher is responsible for publishing events to a NATS messaging system. It uses the `messaging` package to interact with the messaging system. The implementation includes the following features: - Publishing events to NATS using the `Publish` method. - Marshaling events to JSON before publishing. - Setting the message subject and headers based on the event. - Handling errors during publishing. This implementation is built with the `!rabbitmq` build tag, which means it will only be compiled if the `rabbitmq` build tag is not present. The NATS event publisher is part of the Mainflux events package and provides support for the Mainflux NATS events source service functionality. Signed-off-by: Rodney Osodo <socials@rodneyosodo.com> Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com> Signed-off-by: rodneyosodo <blackd0t@protonmail.com> * Add RabbitMQ event publisher implementation This commit adds the implementation of the RabbitMQ event publisher. The RabbitMQ event publisher is responsible for publishing events to a RabbitMQ messaging system. It uses the `messaging` package to interact with the messaging system. The implementation includes the following features: - Publishing events to RabbitMQ using the `Publish` method. - Marshaling events to JSON before publishing. - Setting the message subject and headers based on the event. - Handling errors during publishing. This implementation is built with the `rabbitmq` build tag, which means it will only be compiled if the `rabbitmq` build tag is present. The RabbitMQ event publisher is part of the Mainflux events package and provides support for the Mainflux RabbitMQ events source service functionality. Signed-off-by: Rodney Osodo <socials@rodneyosodo.com> Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com> Signed-off-by: rodneyosodo <blackd0t@protonmail.com> * Add configurable implementation for events store This commit adds a new file `brokers_*.go` which contains the implementation for the different event store. The file includes functions for creating a new publisher and subscriber using different es store. This commit also includes an `init` function that logs a message indicating that the binary was built using the respective package as the events store. The purpose of this commit is to add support for alternative events store. Signed-off-by: Rodney Osodo <socials@rodneyosodo.com> Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com> Signed-off-by: rodneyosodo <blackd0t@protonmail.com> * Fix build flags Signed-off-by: Rodney Osodo <socials@rodneyosodo.com> Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com> Signed-off-by: rodneyosodo <blackd0t@protonmail.com> * Refactor Makefile and Semaphore configuration The Makefile has been refactored to include the `MF_ES_STORE_TYPE` tag in the `go build` command. Additionally, the Semaphore configuration has been updated to include a new task for compiling with Redis as the broker type. This commit addresses the need to compile the codebase with Redis as the event store type and includes the necessary changes in the Makefile and Semaphore configuration. Signed-off-by: Rodney Osodo <socials@rodneyosodo.com> Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com> Signed-off-by: rodneyosodo <blackd0t@protonmail.com> * Reduced due to memory on testing Signed-off-by: Rodney Osodo <socials@rodneyosodo.com> Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com> Signed-off-by: rodneyosodo <blackd0t@protonmail.com> * Fix tests for es Signed-off-by: Rodney Osodo <socials@rodneyosodo.com> Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com> Signed-off-by: rodneyosodo <blackd0t@protonmail.com> * Fix grammar Co-authored-by: Sammy Kerata Oina <44265300+SammyOina@users.noreply.github.com> Signed-off-by: Rodney Osodo <socials@rodneyosodo.com> Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com> Signed-off-by: rodneyosodo <blackd0t@protonmail.com> * Fix linting Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com> Signed-off-by: rodneyosodo <blackd0t@protonmail.com> * feat(docker): update environment variables for message broker The commit updates the environment variable `MF_ES_STORE_TYPE` in the `docker/.env` file. The variable is changed from `${MF_MQTT_BROKER_TYPE}` to `${MF_MESSAGE_BROKER_TYPE}` to accurately reflect the type of message broker being used. This change ensures that the correct message broker is configured for the Event Store. Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com> Signed-off-by: rodneyosodo <blackd0t@protonmail.com> * feat: Update docker environment variables - Removed the unused MF_ES_URL variable in the .env file - Updated the MF_ES_STORE_TYPE and MF_ES_STORE_URL variables in the .env file to match the MF_MESSAGE_BROKER_TYPE and MF_NATS_URL variables respectively Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com> Signed-off-by: rodneyosodo <blackd0t@protonmail.com> * Fix after rebase Signed-off-by: rodneyosodo <blackd0t@protonmail.com> * Add godocs for option parameters for brokers Signed-off-by: rodneyosodo <blackd0t@protonmail.com> * pass by value exchange and prefix names Signed-off-by: rodneyosodo <blackd0t@protonmail.com> * Rename option functions Signed-off-by: rodneyosodo <blackd0t@protonmail.com> * move variables to constants Signed-off-by: rodneyosodo <blackd0t@protonmail.com> * fix: option example comment Signed-off-by: rodneyosodo <blackd0t@protonmail.com> --------- Signed-off-by: Rodney Osodo <socials@rodneyosodo.com> Signed-off-by: Rodney Osodo <28790446+rodneyosodo@users.noreply.github.com> Signed-off-by: rodneyosodo <blackd0t@protonmail.com> Co-authored-by: Sammy Kerata Oina <44265300+SammyOina@users.noreply.github.com> |
||
---|---|---|
.. | ||
api | ||
cache | ||
events | ||
mocks | ||
postgres | ||
standalone | ||
tracing | ||
README.md | ||
doc.go | ||
service.go | ||
service_test.go | ||
things.go |
README.md
Things
Things service provides an HTTP API for managing platform resources: things and channels. Through this API clients are able to do the following actions:
- provision new things
- create new channels
- "connect" things into the channels
For an in-depth explanation of the aforementioned scenarios, as well as thorough understanding of Mainflux, please check out the official documentation.
Configuration
The service is configured using the environment variables presented in the following table. Note that any unset variables will be replaced with their default values.
Variable | Description | Default |
---|---|---|
MF_THINGS_LOG_LEVEL | Log level for Things (debug, info, warn, error) | info |
MF_THINGS_HTTP_HOST | Things service HTTP host | localhost |
MF_THINGS_HTTP_PORT | Things service HTTP port | 9000 |
MF_THINGS_SERVER_CERT | Path to the PEM encoded server certificate file | "" |
MF_THINGS_SERVER_KEY | Path to the PEM encoded server key file | "" |
MF_THINGS_AUTH_GRPC_HOST | Things service gRPC host | localhost |
MF_THINGS_AUTH_GRPC_PORT | Things service gRPC port | 7000 |
MF_THINGS_AUTH_GRPC_SERVER_CERT | Path to the PEM encoded server certificate file | "" |
MF_THINGS_AUTH_GRPC_SERVER_KEY | Path to the PEM encoded server key file | "" |
MF_THINGS_DB_HOST | Database host address | localhost |
MF_THINGS_DB_PORT | Database host port | 5432 |
MF_THINGS_DB_USER | Database user | mainflux |
MF_THINGS_DB_PASS | Database password | mainflux |
MF_THINGS_DB_NAME | Name of the database used by the service | things |
MF_THINGS_DB_SSL_MODE | Database connection SSL mode (disable, require, verify-ca, verify-full) | disable |
MF_THINGS_DB_SSL_CERT | Path to the PEM encoded certificate file | "" |
MF_THINGS_DB_SSL_KEY | Path to the PEM encoded key file | "" |
MF_THINGS_DB_SSL_ROOT_CERT | Path to the PEM encoded root certificate file | "" |
MF_THINGS_CACHE_URL | Cache database URL | redis://localhost:6379/0 |
MF_THINGS_CACHE_KEY_DURATION | Cache key duration in seconds | 3600 |
MF_THINGS_ES_URL | Event store URL | localhost:6379 |
MF_THINGS_ES_PASS | Event store password | "" |
MF_THINGS_ES_DB | Event store instance name | 0 |
MF_THINGS_STANDALONE_ID | User ID for standalone mode (no gRPC communication with users) | "" |
MF_THINGS_STANDALONE_TOKEN | User token for standalone mode that should be passed in auth header | "" |
MF_JAEGER_URL | Jaeger server URL | http://jaeger:14268/api/traces |
MF_AUTH_GRPC_URL | Users service gRPC URL | localhost:7001 |
MF_AUTH_GRPC_TIMEOUT | Users service gRPC request timeout in seconds | 1s |
MF_AUTH_GRPC_CLIENT_TLS | Enable TLS for gRPC client | false |
MF_AUTH_GRPC_CA_CERT | Path to the CA certificate file | "" |
MF_SEND_TELEMETRY | Send telemetry to mainflux call home server. | true |
MF_THINGS_INSTANCE_ID | Things instance ID | "" |
Note that if you want thin gs
service to have only one user locally, you should use MF_THINGS_STANDALONE
env vars. By specifying these, you don't need auth
service in your deployment for users' authorization.
Deployment
The service itself is distributed as Docker container. Check the things
service section in
docker-compose to see how service is deployed.
To start the service outside of the container, execute the following shell script:
# download the latest version of the service
git clone https://github.com/mainflux/mainflux
cd mainflux
# compile the things
make things
# copy binary to bin
make install
# set the environment variables and run the service
MF_THINGS_LOG_LEVEL=[Things log level] \
MF_THINGS_STANDALONE_ID=[User ID for standalone mode (no gRPC communication with auth)] \
MF_THINGS_STANDALONE_TOKEN=[User token for standalone mode that should be passed in auth header] \
MF_THINGS_CACHE_KEY_DURATION=[Cache key duration in seconds] \
MF_THINGS_HTTP_HOST=[Things service HTTP host] \
MF_THINGS_HTTP_PORT=[Things service HTTP port] \
MF_THINGS_HTTP_SERVER_CERT=[Path to server certificate in pem format] \
MF_THINGS_HTTP_SERVER_KEY=[Path to server key in pem format] \
MF_THINGS_AUTH_GRPC_HOST=[Things service gRPC host] \
MF_THINGS_AUTH_GRPC_PORT=[Things service gRPC port] \
MF_THINGS_AUTH_GRPC_SERVER_CERT=[Path to server certificate in pem format] \
MF_THINGS_AUTH_GRPC_SERVER_KEY=[Path to server key in pem format] \
MF_THINGS_DB_HOST=[Database host address] \
MF_THINGS_DB_PORT=[Database host port] \
MF_THINGS_DB_USER=[Database user] \
MF_THINGS_DB_PASS=[Database password] \
MF_THINGS_DB_NAME=[Name of the database used by the service] \
MF_THINGS_DB_SSL_MODE=[SSL mode to connect to the database with] \
MF_THINGS_DB_SSL_CERT=[Path to the PEM encoded certificate file] \
MF_THINGS_DB_SSL_KEY=[Path to the PEM encoded key file] \
MF_THINGS_DB_SSL_ROOT_CERT=[Path to the PEM encoded root certificate file] \
MF_THINGS_CACHE_URL=[Cache database URL] \
MF_THINGS_ES_URL=[Event store URL] \
MF_THINGS_ES_PASS=[Event store password] \
MF_THINGS_ES_DB=[Event store instance name] \
MF_AUTH_GRPC_URL=[Users service gRPC URL] \
MF_AUTH_GRPC_TIMEOUT=[Users service gRPC request timeout in seconds] \
MF_AUTH_GRPC_CLIENT_TLS=[Enable TLS for gRPC client] \
MF_AUTH_GRPC_CA_CERT=[Path to trusted CA certificate file] \
MF_JAEGER_URL=[Jaeger server URL] \
MF_SEND_TELEMETRY=[Send telemetry to mainflux call home server] \
MF_THINGS_INSTANCE_ID=[Things instance ID] \
$GOBIN/mainflux-things
Setting MF_THINGS_CA_CERTS
expects a file in PEM format of trusted CAs. This will enable TLS against the Users gRPC endpoint trusting only those CAs that are provided.
In constrained environments, sometimes it makes sense to run Things service as a standalone to reduce network traffic and simplify deployment. This means that Things service
operates only using a single user and is able to authorize it without gRPC communication with Auth service.
To run service in a standalone mode, set MF_THINGS_STANDALONE_EMAIL
and MF_THINGS_STANDALONE_TOKEN
.
Usage
For more information about service capabilities and its usage, please check out the API documentation.