Mainflux.mainflux/writers/influxdb
Dušan Borovčanin dfa766e013 MF-407 - Values of zero are being omitted (#434)
* Fix empty protobuf values

Update Normalizer service and .proto files. Reader and Writer services
needs to be updated due to message format change.

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Update HTTP adapter to use gogo protobuf

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Update Reader services API

Update API or Reader services to match Message changes due to switching to gogo/proto.

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Update InfluxDB services

Update InfluxDB Reader and Writer services to match new Message format.

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Update MongoDB services

Update MongoDB Reader and Writer services to match new message format.

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Update Cassandra services

Update Cassandra Reader and Writer service to match new Message format.

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Fix InfluxDB Reader test

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Update Makefile and docs accordingly

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Fix possible data race in InfluxDB writer

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Update InfluxDB Writer tests

Raise test coverage.

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Fix InfluxDB reader

Fix wrong ValueSum readings. Upadete tests and raise coverage.

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Update MongoDB services tests

Raise test coverage for MongoDB Reader and Writer services.

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Update Readers API tests

Raise test coverage.

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Raise test coverage

Update Cassandra Reader and Writer services tests.

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Use gogo/protobuf in CoAP adapter

Add gogo/protobuf to Gopkg.toml and update dependencies.
Update Dockerfile to run `make proto`.

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Update Cassandra Reader tests

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Refactor code

Improve code style and comments to improve readability.

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Rename Sum to SumValue

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Rename Values to Value

Since message contains only single value (or possibly no value at all),
`Values` name could be misleading. Rename simple double value from
`Value` to `FloatValue` accordingly.

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Fix InfluxDB Reader logging

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Replace exclusive if statements with switch-case

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>

* Update Cassandra services tests

Signed-off-by: Dusan Borovcanin <dusan.borovcanin@mainflux.com>
2018-11-05 19:18:51 +01:00
..
README.md MF-379 - Log Level Option (#402) 2018-09-26 18:58:51 +02:00
api.go MF-325 - Add SPDX license and copyright headers (#362) 2018-08-26 13:15:48 +02:00
doc.go MF-325 - Add SPDX license and copyright headers (#362) 2018-08-26 13:15:48 +02:00
messages.go MF-407 - Values of zero are being omitted (#434) 2018-11-05 19:18:51 +01:00
messages_test.go MF-407 - Values of zero are being omitted (#434) 2018-11-05 19:18:51 +01:00
setup_test.go MF-325 - Add SPDX license and copyright headers (#362) 2018-08-26 13:15:48 +02:00

README.md

InfluxDB writer

InfluxDB writer provides message repository implementation for InfluxDB.

Configuration

The service is configured using the environment variables presented in the following table. Note that any unset variables will be replaced with their default values.

Variable Description Default
MF_NATS_URL NATS instance URL nats://localhost:4222
MF_INFLUX_WRITER_LOG_LEVEL Log level for InfluxDB writer (debug, info, warn, error) error
MF_INFLUX_WRITER_PORT Service HTTP port 8180
MF_INFLUX_WRITER_BATCH_SIZE Size of the writer points batch 5000
MF_INFLUX_WRITER_BATCH_TIMEOUT Time interval in seconds to flush the batch 1 second
MF_INFLUX_WRITER_DB_NAME InfluxDB database name mainflux
MF_INFLUX_WRITER_DB_HOST InfluxDB host localhost
MF_INFLUX_WRITER_DB_PORT Default port of InfluxDB database 8086
MF_INFLUX_WRITER_DB_USER Default user of InfluxDB database mainflux
MF_INFLUX_WRITER_DB_PASS Default password of InfluxDB user mainflux

Deployment

  version: "2"
  influxdb-writer:
    image: mainflux/influxdb:[version]
    container_name: [instance name]
    expose:
      - [Service HTTP port]
    restart: on-failure
    environment:
      MF_NATS_URL: [NATS instance URL]
      MF_INFLUX_WRITER_LOG_LEVEL: [Influx writer log level]
      MF_INFLUX_WRITER_PORT: [Service HTTP port]
      MF_INFLUX_WRITER_BATCH_SIZE: [Size of the writer points batch]
      MF_INFLUX_WRITER_BATCH_TIMEOUT: [Time interval in seconds to flush the batch]
      MF_INFLUX_WRITER_DB_NAME: [InfluxDB name]
      MF_INFLUX_WRITER_DB_HOST: [InfluxDB host]
      MF_INFLUX_WRITER_DB_PORT: [InfluxDB port]
      MF_INFLUX_WRITER_DB_USER: [InfluxDB admin user]
      MF_INFLUX_WRITER_DB_PASS: [InfluxDB admin password]
    ports:
      - [host machine port]:[configured HTTP port]

To start the service, execute the following shell script:

# download the latest version of the service
go get github.com/mainflux/mainflux


cd $GOPATH/src/github.com/mainflux/mainflux

# compile the influxdb
make influxdb

# copy binary to bin
make install

# Set the environment variables and run the service
MF_NATS_URL=[NATS instance URL] MF_INFLUX_WRITER_LOG_LEVEL=[Influx writer log level] MF_INFLUX_WRITER_PORT=[Service HTTP port] MF_INFLUX_WRITER_BATCH_SIZE=[Size of the writer points batch] MF_INFLUX_WRITER_BATCH_TIMEOUT=[Time interval in seconds to flush the batch] MF_INFLUX_WRITER_DB_NAME=[InfluxDB database name] MF_INFLUX_WRITER_DB_HOST=[InfluxDB database host] MF_INFLUX_WRITER_DB_PORT=[InfluxDB database port] MF_INFLUX_WRITER_DB_USER=[InfluxDB admin user] MF_INFLUX_WRITER_DB_PASS=[InfluxDB admin password] $GOBIN/mainflux-influxdb

Using docker-compose

This service can be deployed using docker containers. Docker compose file is available in <project_root>/docker/addons/influxdb-writer/docker-compose.yml. Besides database and writer service, it contains Grafana platform which can be used for database exploration and data visualization and analytics. In order to run Mainflux InfluxDB writer, execute the following command:

docker-compose -f docker/addons/influxdb-writer/docker-compose.yml up -d

Please note that you need to start core services before the additional ones.

Usage

Starting service will start consuming normalized messages in SenML format.