CCCTC Docs

CCCTC Docs

  • Help

Service Workers

Copyright (c) 2019. California Community Colleges Technology Center
Licensed under the MIT license.
A copy of this license may be found at https://opensource.org/licenses/mit-license.php

The service-workers module is part of the API Gateway

Workflows in service-conductor define the inputs and outputs and aggregation of calls to all microservices. The tasks in service-conductor are defined in service-conductor, but those tasks are executed by external worker threads. These worker threads are referred to below as service workers.

API Gateway

The service-workers module connects service-conductor's workflow engine to the various deployments of backend microservices (i.e. AWS S3 synchronization, and Canvas data exports).

Configuration

Standard Spring profile property files are used.

Running service-workers

Typically you'll run all three components of the API-Gateway stack locally: service-router, service-conductor and service-workers

Instructions and pre-requisites for running these services together can be found in the overall API Gateway documentation here: development-environment

Running the Local stack in 'mixed-mode'

The service-router and service-workers can be run in either an IDE or in Docker.

The table below shows the run options supported by default:

Supported by Default?DockerIDE
Yesservice-workers and service-router
Yesservice-workers and service-router
Yesservice-routerservice-workers
No (local override config required)*service-workersservice-router

*The service-workers poll Conductor via the service-router. In a Docker container, the service-workers connect to the service-router via 'http://service-router...'
When the service-router is in the IDE, the workers would have to access it via localhost. To work around this, add the configuration below to your "application_local.yml file":

  # Conductor URL to poll for work tasks
  conductor:
  # URL to access Conductor (via localhost) when router isn't on Docker network (i.e., running in IDE)
  apiURL: "http://192.168.1.64:10000/conductor/secure/"

Use your machine's IP instead (localhost will not work from within a Docker container).

IntelliJ

The IntelliJ Run Configuration can be configured as:

Main class: org.ccctech.apigateway.serviceworker.ServiceWorkerApplication
VM options: -Dspring.profiles.active=shared,canvasDW,college-adaptor,movies,dev,local -Djasypt.encryptor.password=unlockMe

Spring Profiles Explained

Spring profiles in the service-workers serve two purposes:

  1. Normal spring configuration. These profiles are in src/main/resources and represent the various environments (ci, qa, pilot, prod, dev) as well as one for additional configuration when deployed via docker.
  2. Specifying what services to run with this service-worker. These generally have no configuration of their own (though they could), but instead they are used in the code to enable/disable components. For example, adding college-adaptor to the list of active spring profiles enables the Coordinator for college-adaptor, which "turns on" that worker. You'll see in the code that it is enabled via the annotation @Profile("college-adaptor").

See Service Workers below for more detail on each type of worker.

Docker

Follow the step below get service-workers running locally in Docker using the CCCTC registry locally.

Prerequisite: Follow the instructions on adding your user / pass to the registry here to set your credentials for pulling from the CCCTC private registry.

  1. In your local service-workers directory, use the following command to ensure your local ccctc Docker network is running:

       docker network create ccctc
    
  2. Use the following command if you have NOT built the images locally but want to pull already-built images:

    export DOCKER_REGISTRY=registry.ccctechcenter.org/
    
  3. Login to the CCCTC Docker registry using the following command:

       export DOCKER_REGISTRY=registry.ccctechcenter.org:5000/
    

    Note: The docker-compose.yml file uses the following environment variable:

VariableDescription
DOCKER_REGISTRYURL to remote private registry or "" for local.
  1. From the service-worker root directory, start the Docker container with:

       docker-compose up
    
  2. With the above docker-compose command, you can CTRL-C to stop the processes. To ensure proper cleanup, follow that with:

      docker-compose down
    

Running service-workers From a Local Image

Use the following steps after running the CCCTC Docker registry image to switch to a local environment image in order to make changes to the code locally.

  1. Switch to your local environment by unsetting the DOCKER_REGISTRY with:

    unset DOCKER_REGISTRY
    
  2. Recompile and rebuild the Docker image (necessary before starting the container) with Maven:

     mvn clean install
    
  3. Start the Docker container:

       docker-compose up
    
  4. With the above docker-compose command, you can CTRL-C to stop the processes. To ensure proper cleanup, follow that with:

      docker-compose down
    

Security

The workers are configured to poll service-conductor from the below property.

conductor.apiURL: "http://localhost:10000/conductor/secure/"

Testing

See service-router for testing details through router. All services should be accessed through router, not directly.

Service Workers

In the Netflix API, "coordinators" are components that handle initialization and registration of service workers in Netflix Conductor.

Each coordinator that polls service-conductor is enabled by using a Spring profile.

Listed below are the service workers defined in this module. Each worker can be enabled by activating the associated Spring profile. Therefore when running locally (with above IntelliJ configuration) all profiles are active and all workers will poll Conductor.

Canvas Data Warehouse Data Sync

Enabling Spring profile:

canvasDW

College Adaptor

Enabling Spring profile:

college-adaptor

The College Adaptor worker is a little different than the other workers in that it requires a domain, which identifies the college for which it will process requests. You'll notice in the "dev" profile the domain is "001", which is the mis code that is typically used by our "mock" college adaptor.

Movies

For prototyping purposes, this simple service returns hard coded output and does not proxy requests to another microservice.

Enabling Spring profile:

movies

Student Placement

Enabling Spring profile:

studentPlacement

Limitations

see wiki

Sentry IO

Sentry IO is error tracking software that has been integrated in the service workers project. The integration relies on the existence of an environment variable SENTRY_DSN (the data source name). The data source name is a URL that provides endpoint information to the sentry IO subsystem. When the environment variable is not present, the Sentry IO subsystem is completely disabled.

  • Configuration
  • Running service-workers
    • Running the Local stack in 'mixed-mode'
    • IntelliJ
    • Docker
  • Security
  • Testing
  • <a name="serviceWorkers"></a>Service Workers
    • Canvas Data Warehouse Data Sync
    • College Adaptor
    • Movies
    • Student Placement
  • Limitations
  • Sentry IO