MDM Enable College Adaptor Design (POC)
To "MDM-enable" a service is taken to mean any updates to the service are 'shared' with MDM so that MDM can further share those updates appropriately.
Overview
This is a Proof Of Concept to explore MDM implementation choices with the college adaptor.
Several pieces are still in development or in the process of being finalized in design and implementation. To help shape the final stages of these pieces in progress, this POC will prove some integration choices with the API Gateway.
Business Case
The business case here is typical of a data update and the flow of sharing that update through MDM to downstream systems.
- When a college adaptor (source system) receives an update to a
Person's CCCID attribute (source record), we will notify the MDM framework. With this information, MDM will properly apply governance and other capabilities to share this data update with downstream systems. - A second college adaptor that registers to receive data updates (interested system), will receive an update from MDM containing this new CCCID value.
Technical Requirements
- Ideally, the micro won't have any dependencies or knowledge of MDM. This adheres to a major principal of micro architecture, (viz. "separation of concerns")
API Gateway
The components in the gateway, the service-router, service-conductor, and service-workers will allow us to integrate with MDM and make ZERO changes to the college-adaptor. In fact, all work outlined below involves no Java code and simply leverages the features of the gateway stack.
Service Router
Authorization and service routing (URI) for the MDM college adaptor are described below.
Authorization
Note: The OAuth information below is not new and describes existing functionality.
Proper OAuth 2.0 credentials are required to access all endpoints through the API Gateway.
Calls to update a source adaptor will use existing flows and legacy endpoints. Those endpoints to update the SIS behind the college-adaptor must have the following security scope(s) as documented elsewhere.
- sisdata_W
Note: Data updates from MDM to the interested adaptor will not go through the service-router, but via the eventing system in service-conductor.
URI
Note: The information below is not new and describes existing functionality.
The URI below is exposed to update the source system adaptor:
PATCH /adaptor/{mis}}/mock/colleges/{{mis}}/persons/{{sisPersonId}}
where:
| Label | Description |
|---|---|
| mis | The MIS code of the college to update |
| sisPersonId | The ID of the Person record in the SIS |
This URI will initiate the College Adaptor Workflow below.
Service Conductor
The Service Conductor has the following workflow and task definitions for the MDM college adaptor defined below.
Workflow Definitions
There are two workflows for the MDM college adaptor:
- College Adaptor Workflow
- MDM Workflow
College Adaptor Workflow
Version 1 of the college-adaptorWF will remain the same to support legacy code and allow A-B testing.
A new version 2 of the college-adaptorWF will extend v1 to add an event task to fire when PATCH calls happen for the college adaptor. The Event task is described in Conductor's Event Task.
MDM Workflow
This new workflow is started from the MDM Event Handler.
This workflow is responsible for calling the MDM microservice and passing it details to share/notify the updated Person object.
Note: At this point, exactly how the workflow will call MDM is in development. The top choices discussed are:
- Make a RESTful call to a service that wraps the MDM framework.
- Use an AWS SQS queue to post updates to Conductor and in turn post to the MDM adaptor. This solution may also solve the TBD flow of how to receive updates from MDM.
Task Definitions
The MDM college adaptor has two workflow tasks and one event handler associated with it:
- College Adaptor Intake Task
- MDM event handler
College Adaptor Intake Task
Note: The information below is not new and describes existing functionality.
This task is responsible for supporting all calls to the college adaptor. It is 100% compatible with legacy calls (prior to API Gateway) and will allow future versions and upgrades to be forked into appropriate flows.
The design and implementation details of this new microservice are contained in TODO: link-to-microservice-design-markdown-doc
Input
An example payload to update the cccid field of the specified {{sisPersonId}} in the URI:
{
"cccid": "TEST777"
}
Output
{
"misCode": "001",
"sisPersonId": "person1",
"firstName": "Marge",
"lastName": "Simpson",
"emailAddresses": [
{
"type": "Institution",
"emailAddress": "msimpson@springfield.edu"
},
{
"type": "Personal",
"emailAddress": "msimpson@mail.com"
}
],
"loginId": "msimpson",
"loginSuffix": "springfield.edu",
"cccid": "TEST777"
}
Event Handlers
All event handlers are asynchronous. Events 'fire-and-forget' in the main thread so they have little to no impact on the main flow. In a separate thread, handlers listen for specific events and act on them accordingly.
MDM Event Handler
This handler listens for events produced in College Adaptor Workflow. Events are received by this handler that pass the condition below:
"condition": "$.inbound.httpMethod == '\''PATCH'\'' && $.outbound.statusCodeValue >= 200 && $.outbound.statusCodeValue <= 299 ",
By satisfying this condition, a start_workflow action is performed naming the MDM Workflow to be started.
Service Workers
The service workers for the MDM college adaptor are described below.
College Adaptor
Note: The information below is not new and describes existing functionality.
This worker is required for the College Adaptor Intake Task
This worker is running co-located with the deployment of the college adaptor. From there, the worker calls out to the college adaptor securely on a private network.
Properties
URL of microservice.
Response
An example return from a call to this gateway endpoint will return:
{
"misCode": "001",
"sisPersonId": "person1",
"firstName": "Marge",
"lastName": "Simpson",
"emailAddresses": [
{
"type": "Institution",
"emailAddress": "msimpson@springfield.edu"
},
{
"type": "Personal",
"emailAddress": "msimpson@mail.com"
}
],
"loginId": "msimpson",
"loginSuffix": "springfield.edu",
"cccid": "TEST777"
}
Limitations
We do not define updates from MDM (originated on an upstream system). There are still some design discussions being made in this area and prototyping is in process.
This design does not address updates made at the college, directly into the SIS. This also is too early in the design phase to design a solution around.
Issues outside the college-adaptor such as the issues described below are unresolved:
- Another application such as CCCApply updates some part of a record (e.g., student address), then how are those deltas shared with college-adaptors
- How do we map a person's unique id at one college with that same person at another college? The sisPersonId will be different for the same person at two colleges. However, in the call above, sisPersonId is all we pass for home college's PATCH update.