b3991b8497
* Add starter kit Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add http Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add cmd/main.go Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Remove reference to things from README.md Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add starter kit Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add http Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add cmd/main.go Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Remove reference to things from README.md Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Fix env vars in README.md Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Fix env vars in README.md Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Rename kit to mfxkit Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Rename kit to mfxkit Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add docker compose related files Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add docker compose related files Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add Dockerfile to mfxkit Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add Dockerfile to mfxkit Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add twin service to docker-compose.yml Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add twin service to docker-compose.yml Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add mongo db connection Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add mongo db connection Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add TwinRepository mockup Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Fix docker env vars Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Fix docker env vars Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add twins repo mongodb routines Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add twins repo mongodb routines Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add mongo db docker test suite Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add mongo db docker test suite Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add idp and toDBTwin Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add idp and toDBTwin Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add test cases to repo twin save test Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add test cases to repo twin save test Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add users grpc Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add users grpc Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add methods and tests for udpate and update key Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add methods and tests for udpate and update key Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add methods and tests for remove twin Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add methods and tests for remove twin Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add svc methods to loggin and metrics Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add svc methods to loggin and metrics Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add AddTwin endpoint Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add AddTwin endpoint Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add update endpoints Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add update endpoints Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add view and remove endpoints Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add view and remove endpoints Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add twin repo mock Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add twin repo mock Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add owner arg to twins repo methods Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add owner arg to twins repo methods Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add mock idp service Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add mock users service Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add tests for service Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add tests for service Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add mqtt client to twins service Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add mqtt client to twins service Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add protocol to mqtt string var Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add protocol to mqtt string var Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add time and attributes related data to Twin struct Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add time and attributes related data to Twin struct Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add mqtt publish JSON serialized twin to service Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add mqtt publish JSON serialized twin to service Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add endpoint tests for save and update twin Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add endpoint tests for save and update twin Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add endpoint tests for update key and view thing Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add endpoint tests for update key and view thing Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Fix test error for view twin Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Fix test error for view twin Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add endpoint tests for remove twin Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add endpoint tests for remove twin Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add NATS client Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add NATS client Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add normalizer to nats Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add normalizer to nats Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Refactor nats publish() and remove normalizer Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Refactor nats publish() and remove normalizer Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add ListThingsByChannel() and RetrieveByChannel() Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add ListThingsByChannel() and RetrieveByChannel() Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add Twin struct directly in mongodb Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add Twin struct directly in mongodb Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Change limit type to uint64 Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Change limit type to uint64 Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Compose nats subject from msg ch and subtopic Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Compose nats subject from msg ch and subtopic Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add mqtt notifs for thing creation and key update Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add mqtt notifs for thing creation and key update Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add RetrieveAll to twinRepository and ListTwins to service Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add RetrieveAll to twinRepository and ListTwins to service Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add tests for RetrieveAll and ListTwins Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add tests for RetrieveAll and ListTwins Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Fix Service interface error Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Remove test Ping endpoint Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add mqtt topic to service config Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Remove UpdateKey request and add fields to Twin related requests Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add Attributes and State add and view Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add ListTwins endpoint Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Fix service and database tests Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add ListTwinsByThing endpoint Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Remove twin directory Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add States and Definitions to Twin Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add defer publish to AddTwin Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add refactored and deferred mqtt client publish to Service methods Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add paho subservice Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add paho client to nats Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add mqtt publish and state update Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add Publish wrapper Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Use paho Publish wrapper Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Fix tests Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Save states in separate collection Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Replace []byte payload by []map[string]interface{} Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add offset to PageMetadata Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Rename TwinsPage to Page Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add StateRepository Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add StateRepository, mongodb StateRepository adapter and ListStates endpoint Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add RetrieveLast twin to states repository Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Parse def in nats to update state attrib Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add prepareState() helper Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Replace list things by id by view thing by id Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add state repo mock Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Fix service tests Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Fix mongo twins repo tests and remove owner from RetrieveByID params Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Fix mongo twins repo tests and remove owner from Remove params Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Replace ChannelID in Attribute by Channel Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Fix endpoint tests for add and update twin Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Fix endpoint tests for view and remove twin Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Use new auth service Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Fix mocks auth and mongodb tests Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Fix licensing info Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Fix 'for for' Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Replace short dec by var for default vals and add string map for mqtt op info Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Use Record from mainflux/senml and rename broker to nats in main.go Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Remove Key from Twin Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Remove redundant id uniqueness check from mongodb Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add attrib name to attrib update info Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Fix typos and remove isZeroOfUnderlyingType() helper Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Rename paho to mqtt Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Fix env vars in main.go and README.md Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Move twins/api/twins to twins/api and rename twins- prefix to tw- in main.go Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Remove flows for manual testing and revert to master docker-compose.yml Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Remove publish from nats and fix tests by updating package names Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Rename users.go to authn.go Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Fix crud op names Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Change id to twinID Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Inline if err != nil and change <prefix>ID to <prefix>_id Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Fix metadata db search test Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Rename mqtt and nats related files to publisher and subscriber respectively Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Move save state logic from nats to service Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Fix endpoint tests Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Remove close check from main.go Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Complete deploy section Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add unit tests for states Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Remove debugging artefacts and unneeded comments to exported funcs Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> * Add port num to defAuthnURL Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> |
||
---|---|---|
.. | ||
api | ||
mocks | ||
mongodb | ||
mqtt | ||
nats | ||
uuid | ||
README.md | ||
doc.go | ||
idp.go | ||
service.go | ||
service_test.go | ||
states.go | ||
twins.go |
README.md
Twins
Service twins is used for CRUD and update of digital twins. Twin is a semantic representation of a real world entity, be it device, application or something else. It holds the sequence of attribute based definitions of a real world thing and refers to the time series of definition based states that hold the historical data about the represented real world thing.
Configuration
The service is configured using the environment variables presented in the following table. Note that any unset variables will be replaced with their default values.
Variable | Description | Default |
---|---|---|
MF_TWINS_LOG_LEVEL | Log level for twin service (debug, info, warn, error) | error |
MF_TWINS_HTTP_PORT | Twins service HTTP port | 9021 |
MF_TWINS_SERVER_CERT | Path to server certificate in PEM format | |
MF_TWINS_SERVER_KEY | Path to server key in PEM format | |
MF_JAEGER_URL | Jaeger server URL | |
MF_TWINS_DB_NAME | Database name | mainflux |
MF_TWINS_DB_HOST | Database host address | localhost |
MF_TWINS_DB_PORT | Database host port | 27017 |
MF_TWINS_SINGLE_USER_EMAIL | User email for single user mode (no gRPC communication with users) | |
MF_TWINS_SINGLE_USER_TOKEN | User token for single user mode that should be passed in auth header | |
MF_TWINS_CLIENT_TLS | Flag that indicates if TLS should be turned on | false |
MF_TWINS_CA_CERTS | Path to trusted CAs in PEM format | |
MF_TWINS_MQTT_URL | Mqtt broker URL for twin CRUD and states update notifications | tcp://localhost:1883 |
MF_TWINS_THING_ID | ID of thing representing twins service & mqtt user | |
MF_TWINS_THING_KEY | Key of thing representing twins service & mqtt pass | |
MF_TWINS_CHANNEL_ID | Mqtt notifications topic | |
MF_NATS_URL | Mainflux NATS broker URL | nats://127.0.0.1:4222 |
MF_AUTHN_GRPC_PORT | Authn service gRPC port | 8181 |
MF_AUTHN_TIMEOUT | Authn gRPC request timeout in seconds | 1 |
MF_AUTHN_URL | Authn service URL | localhost:8181 |
Deployment
The service itself is distributed as Docker container. The following snippet provides a compose file template that can be used to deploy the service container locally:
version: "3"
services:
twins:
image: mainflux/twins:[version]
container_name: [instance name]
ports:
- [host machine port]:[configured HTTP port]
environment:
MF_TWINS_LOG_LEVEL: [Twins log level]
MF_TWINS_HTTP_PORT: [Service HTTP port]
MF_TWINS_SERVER_CERT: [String path to server cert in pem format]
MF_TWINS_SERVER_KEY: [String path to server key in pem format]
MF_JAEGER_URL: [Jaeger server URL]
MF_TWINS_DB_NAME: [Database name]
MF_TWINS_DB_HOST: [Database host address]
MF_TWINS_DB_PORT: [Database host port]
MF_TWINS_SINGLE_USER_EMAIL: [User email for single user mode]
MF_TWINS_SINGLE_USER_TOKEN: [User token for single user mode]
MF_TWINS_CLIENT_TLS: [Flag that indicates if TLS should be turned on]
MF_TWINS_CA_CERTS: [Path to trusted CAs in PEM format]
MF_TWINS_MQTT_URL: [Mqtt broker URL for twin CRUD and states]
MF_TWINS_THING_ID: [ID of thing representing twins service]
MF_TWINS_THING_KEY: [Key of thing representing twins service]
MF_TWINS_CHANNEL_ID: [Mqtt notifications topic]
MF_NATS_URL: [Mainflux NATS broker URL]
MF_AUTHN_GRPC_PORT: [Authn service gRPC port]
MF_AUTHN_TIMEOUT: [Authn gRPC request timeout in seconds]
MF_AUTHN_URL: [Authn service URL]
To start the service outside of the container, execute the following shell script:
# download the latest version of the service
go get github.com/mainflux/mainflux
cd $GOPATH/src/github.com/mainflux/mainflux
# compile the twins
make twins
# copy binary to bin
make install
# set the environment variables and run the service
MF_TWINS_LOG_LEVEL: [Twins log level] MF_TWINS_HTTP_PORT: [Service HTTP port] MF_TWINS_SERVER_CERT: [String path to server cert in pem format] MF_TWINS_SERVER_KEY: [String path to server key in pem format] MF_JAEGER_URL: [Jaeger server URL] MF_TWINS_DB_NAME: [Database name] MF_TWINS_DB_HOST: [Database host address] MF_TWINS_DB_PORT: [Database host port] MF_TWINS_SINGLE_USER_EMAIL: [User email for single user mode] MF_TWINS_SINGLE_USER_TOKEN: [User token for single user mode] MF_TWINS_CLIENT_TLS: [Flag that indicates if TLS should be turned on] MF_TWINS_CA_CERTS: [Path to trusted CAs in PEM format] MF_TWINS_MQTT_URL: [Mqtt broker URL for twin CRUD and states] MF_TWINS_THING_ID: [ID of thing representing twins service] MF_TWINS_THING_KEY: [Key of thing representing twins service] MF_TWINS_CHANNEL_ID: [Mqtt notifications topic] MF_NATS_URL: [Mainflux NATS broker URL] MF_AUTHN_GRPC_PORT: [Authn service gRPC port] MF_AUTHN_TIMEOUT: [Authn gRPC request timeout in seconds] MF_AUTHN_URL: [Authn service URL] $GOBIN/mainflux-twins
Usage
For more information about service capabilities and its usage, please check out the API documentation.