Mainflux.mainflux/consumers/writers/influxdb
Manuel Imperiale b18c9e79dc
NOISSUE - Add timestamp transformation rules for specifc JSON fields (#1514)
* NOISSUE - Add timestamp transformation rules for specifc JSON fields

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Add tests and defaults

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix reviews

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix Created

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix READMEs

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix Config file

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Improve the timestamp Config structure

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix typos

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix smpp

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix reviews

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Rm duplicated Unmarshal

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Extract the error var

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix review

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Return error if time field transformation fails

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>
2021-12-14 11:17:23 +01:00
..
README.md NOISSUE - Add timestamp transformation rules for specifc JSON fields (#1514) 2021-12-14 11:17:23 +01:00
consumer.go MF-1421 - Make flattening of JSON transformer only available on InfluxDB (#1432) 2021-07-22 11:20:47 +02:00
consumer_test.go MF-1421 - Make flattening of JSON transformer only available on InfluxDB (#1432) 2021-07-22 11:20:47 +02:00
doc.go NOISSUE - Switch to Consumers interface (#1316) 2021-01-11 23:55:34 +01:00
fields.go NOISSUE - Switch to Consumers interface (#1316) 2021-01-11 23:55:34 +01:00
setup_test.go MF-1378 - Update dependencies (#1379) 2021-05-20 20:53:56 +02:00
tags.go NOISSUE - Switch to Consumers interface (#1316) 2021-01-11 23:55:34 +01:00

README.md

InfluxDB writer

InfluxDB writer provides message repository implementation for InfluxDB.

Configuration

The service is configured using the environment variables presented in the following table. Note that any unset variables will be replaced with their default values.

Variable Description Default
MF_NATS_URL NATS instance URL nats://localhost:4222
MF_INFLUX_WRITER_LOG_LEVEL Log level for InfluxDB writer (debug, info, warn, error) error
MF_INFLUX_WRITER_PORT Service HTTP port 8180
MF_INFLUX_WRITER_DB_HOST InfluxDB host localhost
MF_INFLUXDB_PORT Default port of InfluxDB database 8086
MF_INFLUXDB_ADMIN_USER Default user of InfluxDB database mainflux
MF_INFLUXDB_ADMIN_PASSWORD Default password of InfluxDB user mainflux
MF_INFLUXDB_DB InfluxDB database name mainflux
MF_INFLUX_WRITER_CONFIG_PATH Config file path with NATS subjects list, payload type and content-type /configs.toml

Deployment

The service itself is distributed as Docker container. Check the influxdb-writer service section in docker-compose to see how service is deployed.

To start the service, execute the following shell script:

# download the latest version of the service
git clone https://github.com/mainflux/mainflux

cd mainflux

# compile the influxdb
make influxdb

# copy binary to bin
make install

# Set the environment variables and run the service
MF_NATS_URL=[NATS instance URL] \
MF_INFLUX_WRITER_LOG_LEVEL=[Influx writer log level] \
MF_INFLUX_WRITER_PORT=[Service HTTP port] \
MF_INFLUXDB_DB=[InfluxDB database name] \
MF_INFLUX_WRITER_DB_HOST=[InfluxDB database host] \
MF_INFLUXDB_PORT=[InfluxDB database port] \
MF_INFLUXDB_ADMIN_USER=[InfluxDB admin user] \
MF_INFLUXDB_ADMIN_PASSWORD=[InfluxDB admin password] \
MF_INFLUX_WRITER_CONFIG_PATH=[Config file path with NATS subjects list, payload type and content-type] \
$GOBIN/mainflux-influxdb

Using docker-compose

This service can be deployed using docker containers. Docker compose file is available in <project_root>/docker/addons/influxdb-writer/docker-compose.yml. Besides database and writer service, it contains Grafana platform which can be used for database exploration and data visualization and analytics. In order to run Mainflux InfluxDB writer, execute the following command:

docker-compose -f docker/addons/influxdb-writer/docker-compose.yml up -d

Please note that you need to start core services before the additional ones.

Usage

Starting service will start consuming normalized messages in SenML format.