API Gateway Documentation

Resources here contain the technical documentation relating to the API Gateway stack.

The intention is that every service exposed through the gateway has its API Gateway design and implementation documented here.

Design vs Implementation Documentation Guidelines

The service design documentation addresses what the services will do, as driven by the project requirements.

The service design should:

  • be the first step in describing how the microservice will perform its function
  • be high-level enough that the implementation (code) can change and the fundamental design is still intact
  • not include details such as package names, classes, or methods
  • describe (at a high level) how the microservice does X,Y, and Z

The back-end microservices' implementation and configuration details are documented in their separate Git repos. The backend microservices' public interface and exposure of those microservices will be documented here. As back-end micros change, the public endpoints in the gateway should be much more stable and abstract away changes to the back end (see API Gateway).

Release and Deploy Process

We've outlined the release and deploy strategy in SuperGLUE-release.

API Gateway Testing

Integration and end-to-end testing are described in the testing document.

Setting up your development environment

Setting up your environment for develop can be found here: development-environment

Design Documentation

Design documentation for the API Gateway centers around public endpoints, non-public reusable services, and Netflix Conductor.

Gateway Public Endpoints

Publicly-available services or endpoints and their design documentation are listed below.

POC / Prototype Public Endpoints

The prototype endpoints listed below are likely to change and are to be considered work in progress.

Internal (Non-Public) Shared Services

The internal/non-public services listed below aren't called directly from external clients, but are re-usable components used in various routes through the gateway.

Conductor Plugins

Netflix Conductor provides a plugin architecture using Google Guice. This allows us to add our own plugins and extend the base functionality.

Adding new plugins to conductor involves a CSV of classes that implement a guice Module and are specified with a property:

conductor.additional.modules=org.ccctech.apigateway.conductor.SpringBootstrap,org.ccctech.apigateway.conductor.modules.sqs.SQSModule

We have not contributed to Conductor public. However, by providing our own plugins and packaging things into a custom Docker image built FROM (in Docker terms) the Netflix base image, we are able to extend their functionality. We've been able to upgrade Conductor fairly easily with this approach, though sometimes there are changes made to the base source that breaks backward compatibility and requires some tweaking.

Our build creates a custom image that includes our own JAR with plugin code and some additional configuration files and settings.

Amazon Simple Queue System

This plugin allows integration with Amazon SQS. Design details are detailed here.

JSON Transformations

This Conductor System Task allows manipulating JSON input/output with an XPath type syntax. Design details are detailed here

Value Mapper

This Conductor System Task allows manipulating values by way of switch/case or java methods. Design details defailted here

Status Task

This Conductor System Task allows manipulating the overall status of the workflow based on a DECISION or condition. Design details defailted here

START_WORKFLOW System Task

This task starts another workflow by passing the workflow content as the task's inputParameters.

See System Task

Unlike default options for starting workflows, this allows setting a domain on the workflow being started using taskToDomain.

See Task Domains

An example call to get a list of terms from adaptor 001.

            {
                "name": "start-workflow",
                "taskReferenceName": "adaptor",
                "type": "START_WORKFLOW",
                "inputParameters": {
                    "name": "college-adaptorWF",
                    "version" : "1",
                    "input" : {
                        "httpMethod": "GET",
                        "misCode": "001",
                        "parameterMap": {
                            "mis": [
                                "001"
                            ]
                        },
                        "uri": "terms"
                    },
                    "taskToDomain": {
                        "*": "001"
                    },
                    "schemaVersion": 2
                }
            }

START_WORKFLOW_IF System Task

As an extension to the above START_WORKFLOW System task, this custom task will start a workflow if, and only if, a set of conditional statements are met.

An example:

  {
    "name": "dev-startworkflow-if",
    "description": "dev test for start workflow if logic",
    "version": 1,
    "inputParameters": [
      "student-application"
    ],
    "tasks": [
      {
        "name": "start-workflow",
        "taskReferenceName": "start-if",
        "type": "START_WORKFLOW_IF",
        "inputParameters": {
          "condition" : [
            {
              "is-blank" : "${workflow.input.student-application.ccgiToken}"
            }
            ,{
              "is-not-blank" : "${workflow.input.student-application.ssid}"
            }
          ],
          "name": "debugWF",
          "version": 0,
          "input": {
            "message": "WF stub called when START_WORKFLOW_IF condition logic met",
            "student-application": "${workflow.input.student-application}"
          }
        }
      }
    ],
    "schemaVersion": 2
  }

In the above:

  • "is-blank" will be true if the value (here using an input parameter) is blank, empty or null.
  • "is-not-blank" is true if the value is non null and has a length > 0

When all conditions are true, the workflow (here debugWF) is started. If not all conditions are evaluated as true, the 'parent' workflow (here 'dev-startworkflow-if') is completed successfully, but the child (debugWF) is not executed.

In-Memory Cache

As a Conductor System Task, it runs in the server VM. The task allows storing arbitrary data into an in-memory cache as well as searching the cache. Details defailted here

Securing Conductor UI Access

Viewing or running actions on workflows must be protected to allow only authorized user access. This is a customization of the Docker image to overlay security.

The Authentication docs outline the enforced security details around accessing Conductor.