Service Workers
Copyright (c) 2019. California Community Colleges Technology Center
Licensed under the MIT license.
A copy of this license may be found at https://opensource.org/licenses/mit-license.php
The service-workers module is part of the API Gateway
Workflows in service-conductor define the inputs and outputs and aggregation of calls to all microservices. The tasks in service-conductor are defined in service-conductor, but those tasks are executed by external worker threads. These worker threads are referred to below as service workers.
The service-workers module connects service-conductor's workflow engine to the various deployments of backend microservices (i.e. AWS S3 synchronization, and Canvas data exports).
Configuration
Standard Spring profile property files are used.
Running service-workers
Typically you'll run all three components of the API-Gateway stack locally: service-router, service-conductor and service-workers
Instructions and pre-requisites for running these services together can be found in the overall API Gateway documentation here: development-environment
Running the Local stack in 'mixed-mode'
The service-router and service-workers can be run in either an IDE or in Docker.
The table below shows the run options supported by default:
Supported by Default? | Docker | IDE |
---|---|---|
Yes | service-workers and service-router | |
Yes | service-workers and service-router | |
Yes | service-router | service-workers |
No (local override config required)* | service-workers | service-router |
*The service-workers poll Conductor via the service-router. In a Docker container, the service-workers connect to the service-router via 'http://service-router...'
When the service-router is in the IDE, the workers would have to access it via localhost. To work around this, add the configuration below to your
"application_local.yml file":
# Conductor URL to poll for work tasks
conductor:
# URL to access Conductor (via localhost) when router isn't on Docker network (i.e., running in IDE)
apiURL: "http://192.168.1.64:10000/conductor/secure/"
Use your machine's IP instead (localhost will not work from within a Docker container).
IntelliJ
The IntelliJ Run Configuration can be configured as:
Main class: org.ccctech.apigateway.serviceworker.ServiceWorkerApplication
VM options: -Dspring.profiles.active=shared,canvasDW,college-adaptor,movies,dev,local -Djasypt.encryptor.password=unlockMe
Spring Profiles Explained
Spring profiles in the service-workers serve two purposes:
- Normal spring configuration. These profiles are in src/main/resources and represent the various environments (ci, qa, pilot, prod, dev) as well as one for additional configuration when deployed via docker.
- Specifying what services to run with this service-worker. These generally have no configuration of their own (though
they could), but instead they are used in the code to enable/disable components. For example, adding college-adaptor
to the list of active spring profiles enables the Coordinator for college-adaptor, which "turns on" that worker. You'll
see in the code that it is enabled via the annotation
@Profile("college-adaptor")
.
See Service Workers
below for more detail on each type of worker.
Docker
Follow the step below get service-workers running locally in Docker using the CCCTC registry locally.
Prerequisite: Follow the instructions on adding your user / pass to the registry here to set your credentials for pulling from the CCCTC private registry.
In your local service-workers directory, use the following command to ensure your local ccctc Docker network is running:
docker network create ccctc
Use the following command if you have NOT built the images locally but want to pull already-built images:
export DOCKER_REGISTRY=registry.ccctechcenter.org/
Login to the CCCTC Docker registry using the following command:
export DOCKER_REGISTRY=registry.ccctechcenter.org:5000/
Note: The docker-compose.yml file uses the following environment variable:
Variable | Description |
---|---|
DOCKER_REGISTRY | URL to remote private registry or "" for local. |
From the service-worker root directory, start the Docker container with:
docker-compose up
With the above docker-compose command, you can CTRL-C to stop the processes. To ensure proper cleanup, follow that with:
docker-compose down
Running service-workers From a Local Image
Use the following steps after running the CCCTC Docker registry image to switch to a local environment image in order to make changes to the code locally.
Switch to your local environment by unsetting the DOCKER_REGISTRY with:
unset DOCKER_REGISTRY
Recompile and rebuild the Docker image (necessary before starting the container) with Maven:
mvn clean install
Start the Docker container:
docker-compose up
With the above docker-compose command, you can CTRL-C to stop the processes. To ensure proper cleanup, follow that with:
docker-compose down
Security
The workers are configured to poll service-conductor from the below property.
conductor.apiURL: "http://localhost:10000/conductor/secure/"
Testing
See service-router for testing details through router. All services should be accessed through router, not directly.
Service Workers
In the Netflix API, "coordinators" are components that handle initialization and registration of service workers in Netflix Conductor.
Each coordinator that polls service-conductor is enabled by using a Spring profile.
Listed below are the service workers defined in this module. Each worker can be enabled by activating the associated Spring profile. Therefore when running locally (with above IntelliJ configuration) all profiles are active and all workers will poll Conductor.
Canvas Data Warehouse Data Sync
Enabling Spring profile:
canvasDW
College Adaptor
Enabling Spring profile:
college-adaptor
The College Adaptor worker is a little different than the other workers in that it requires a domain, which identifies the college for which it will process requests. You'll notice in the "dev" profile the domain is "001", which is the mis code that is typically used by our "mock" college adaptor.
Movies
For prototyping purposes, this simple service returns hard coded output and does not proxy requests to another microservice.
Enabling Spring profile:
movies
Student Placement
Enabling Spring profile:
studentPlacement
Limitations
Sentry IO
Sentry IO is error tracking software that has been integrated in the service workers project. The integration relies on the existence of an environment variable SENTRY_DSN (the data source name). The data source name is a URL that provides endpoint information to the sentry IO subsystem. When the environment variable is not present, the Sentry IO subsystem is completely disabled.