Mainflux.mainflux/readers/postgres
Mirko Teodorovic b91024465c
MF-1179 - Add a certificate service and certs endpoint to SDK (#1188)
* adding certificate issuing

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* adding cert endpoint

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* update envs

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* update envs

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* move certs creation to sdk

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* move certs creation to sdk

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* move certs creation to sdk

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* fix env vars

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* add comment

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* update sdk

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* fix vars

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* add volumes

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* fix merge config for int

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* remove env

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* fix error handling

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* add cert test, change receiver to pointer

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* add docs

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* fix var naming

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* correct error naming

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* adding certs service

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* add certs

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* add certs

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* change func receiever

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* add default cert issue method

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* add config

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* small fix

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* remove some testing code

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* add cert issue

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* add vault api client

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* additional endpoints

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* add swagger for certs

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* remove certs from provision

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* clean provision from certs

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* add list certificates endpoint

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* add vault api in vendor

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* add certs

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* add revoke, fix bugs

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* fix sdk for certs

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* minor changes, add env, doc

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* minor changes, add env, doc

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* minor changes, add env, doc

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* small changes

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* remove CA for signing from provision

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* add docker file for certs

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* fix mock sdk

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* add line

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* fix RevokeCert

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* renam ENV

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* remove tests temporarily

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* fix naming

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* renam vars

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* add cli for issue cert

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* add cli for issue cert

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* add cli for issue cert

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* add cli for issue cert

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* remove not needed envs

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* fix linter errors, add cli

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* fix linter errors, add cli, var rename

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* fix reviews, add viewcert, fix view all certs

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* remove view cert, as it will be retrieved from PKI

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* change endpoints

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* add default env val

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* remove some errors

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* refactor, make wrapper lib for vault

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* refactor, make wrapper lib for vault

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* refactor, make wrapper lib for vault

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* fix revoking

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* refactor, make wrapper lib for vault

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* update vendor

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* fix comment

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* add comments

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* remove unused

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* remove unused field

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* update vendor

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* refactor pki

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* refactor pki

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* refactor pki, update vendor

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* refactor pki

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* fix comment

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* minor fix

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* remove methods, use fields

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* fix comments and package desc

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* fix comments and package desc

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>
2020-07-21 12:53:21 +02:00
..
README.md MF-1025 - timeout env in sec, use parseduration (#1206) 2020-06-08 18:17:16 +02:00
doc.go MF-788 - Remove date and minimize copyright comments (#876) 2019-10-07 16:14:47 +02:00
init.go MF-950 - Runtime error in normalizer - CBOR SenML (#974) 2019-11-29 20:47:28 +01:00
messages.go Fix typo in error messages (#1193) 2020-06-03 17:34:01 +02:00
messages_test.go MF-1190 - Add pkg for library packages (#1191) 2020-06-03 15:16:19 +02:00
setup_test.go MF-1179 - Add a certificate service and certs endpoint to SDK (#1188) 2020-07-21 12:53:21 +02:00

README.md

Postgres reader

Postgres reader provides message repository implementation for Postgres.

Configuration

The service is configured using the environment variables presented in the following table. Note that any unset variables will be replaced with their default values.

Variable Description Default
MF_POSTGRES_READER_LOG_LEVEL Service log level debug
MF_POSTGRES_READER_PORT Service HTTP port 8180
MF_POSTGRES_READER_CLIENT_TLS TLS mode flag false
MF_POSTGRES_READER_CA_CERTS Path to trusted CAs in PEM format
MF_POSTGRES_READER_DB_HOST Postgres DB host postgres
MF_POSTGRES_READER_DB_PORT Postgres DB port 5432
MF_POSTGRES_READER_DB_USER Postgres user mainflux
MF_POSTGRES_READER_DB_PASS Postgres password mainflux
MF_POSTGRES_READER_DB Postgres database name messages
MF_POSTGRES_READER_DB_SSL_MODE Postgres SSL mode disabled
MF_POSTGRES_READER_DB_SSL_CERT Postgres SSL certificate path ""
MF_POSTGRES_READER_DB_SSL_KEY Postgres SSL key ""
MF_POSTGRES_READER_DB_SSL_ROOT_CERT Postgres SSL root certificate path ""
MF_JAEGER_URL Jaeger server URL localhost:6831
MF_THINGS_AUTH_GRPC_URL Things service Auth gRPC URL localhost:8181
MF_THINGS_AUTH_GRPC_TIMEOUT Things service Auth gRPC timeout in seconds 1s

Deployment

  version: "3.7"
  postgres-writer:
    image: mainflux/postgres-writer:[version]
    container_name: [instance name]
    depends_on:
      - postgres
      - nats
    restart: on-failure
    environment:
      MF_NATS_URL: [NATS instance URL]
      MF_POSTGRES_READER_LOG_LEVEL: [Service log level]
      MF_POSTGRES_READER_PORT: [Service HTTP port]
      MF_POSTGRES_READER_DB_HOST: [Postgres host]
      MF_POSTGRES_READER_DB_PORT: [Postgres port]
      MF_POSTGRES_READER_DB_USER: [Postgres user]
      MF_POSTGRES_READER_DB_PASS: [Postgres password]
      MF_POSTGRES_READER_DB: [Postgres database name]
      MF_POSTGRES_READER_DB_SSL_MODE: [Postgres SSL mode]
      MF_POSTGRES_READER_DB_SSL_CERT: [Postgres SSL cert]
      MF_POSTGRES_READER_DB_SSL_KEY: [Postgres SSL key]
      MF_POSTGRES_READER_DB_SSL_ROOT_CERT: [Postgres SSL Root cert]
      MF_JAEGER_URL: [Jaeger server URL]
      MF_THINGS_AUTH_GRPC_URL: [Things service Auth gRPC URL]
      MF_THINGS_AUTH_GRPC_TIMEOUT: [Things service Auth gRPC request timeout in seconds]
    ports:
      - 8180:8180
    networks:
      - docker_mainflux-base-net

To start the service, execute the following shell script:

# download the latest version of the service
git clone https://github.com/mainflux/mainflux

cd mainflux

# compile the postgres writer
make postgres-writer

# copy binary to bin
make install

# Set the environment variables and run the service
MF_POSTGRES_READER_LOG_LEVEL=[Service log level] \
MF_POSTGRES_READER_PORT=[Service HTTP port] \
MF_POSTGRES_READER_CLIENT_TLS =[TLS mode flag] \
MF_POSTGRES_READER_CA_CERTS=[Path to trusted CAs in PEM format] \
MF_POSTGRES_READER_DB_HOST=[Postgres host] \
MF_POSTGRES_READER_DB_PORT=[Postgres port] \
MF_POSTGRES_READER_DB_USER=[Postgres user] \
MF_POSTGRES_READER_DB_PASS=[Postgres password] \
MF_POSTGRES_READER_DB=[Postgres database name] \
MF_POSTGRES_READER_DB_SSL_MODE=[Postgres SSL mode] \
MF_POSTGRES_READER_DB_SSL_CERT=[Postgres SSL cert] \
MF_POSTGRES_READER_DB_SSL_KEY=[Postgres SSL key] \
MF_POSTGRES_READER_DB_SSL_ROOT_CERT=[Postgres SSL Root cert] \
MF_JAEGER_URL=[Jaeger server URL] \
MF_THINGS_AUTH_GRPC_URL=[Things service Auth GRPC URL] \
MF_THINGS_AUTH_GRPC_TIMEOUT=[Things service Auth gRPC request timeout in seconds] \
$GOBIN/mainflux-postgres-reader

Usage

Starting service will start consuming normalized messages in SenML format.