This page is about the design of a notification system that could be shared between TTAT and Workspaces
Workspaces Notification Service
The current (WS sprint 5-6) version of our notification service is quite simple. There is a REST API for creating notification templates, which consist of a name and a Mustache template. When you want to send a notification, you post to a REST API with the name of the notification you want to send and JSON arguments. One of the JSON arguments is receiver_email and that becomes the recipient. The JSON arguments in total are the context for the Mustache template render. The result of that render is emailed to the recipient.
This system has a significant benefit, in that notification templates can change while the application is online, without waiting for redeployments. This comes with the extra cost that one must be sure that the templates are going to render without error.
Future directions
The following are ideas for directions that the notification service should probably evolve towards.
Notification previews
If we allow system managers to modify the templates, we are going to need to provide them with some way of knowing if the template is valid and going to produce the expected result when the expected notifications arrive. A preview mechanism, perhaps with the addition of a JSON schema, could work for this.
Authentication
The notification service is currently protected by being hidden behind other services in a Docker configuration. If we want to open this up to sharing, we will want to arrange for application authorization, so that random users are not able to send REST calls to cause notifications to be sent. Authentication and authorization is a larger and separate issue (see Authentication, Authorization and Allocation (A3) System) but having well-defined and in-common concepts of principal and whatever else might go with it, would be a useful thing to make a formal part of an API here. In other words, "I am some application, please send this notification on behalf of some user."
Non-email protocols
For internal stuff, Slack notifications are useful. Other possible protocols for notifications include SMS, or server-sent events for HTTP. The current system assumes it is sending email, but that fact is mostly hidden from the notification sender. Maintaining that separation will make it easy to start sending notification over other protocols, as configured in the notification service.
Keeping this configuration in the notification service and outside the applications that need to send notifications opens up the possibility of changing how notifications are sent at runtime, without waiting for an application deployment.
Multiple notifications
It would not be hard to introduce a 1:Many relationship between notifications and templates/messages. This could enable sending multiple notifications in response to one notification event. In workspaces, an example of this might be sending one note to the QA team for a capability reaching that step while also sending a notification to the user that the processing has entered the quality assurance state. The workspaces application wouldn't need to intentionally send two notifications, but two would get sent out by the notification system.
Implementation Notes
A more rigorous conception:
Design questions
Consider the three protocols SMS, Email and Slack. Of these, one is simply body text (SMS); the other two have additional message attributes. Email has arbitrary headers, but some of those headers are essentially mandatory (From, Subject). Slack has a mandatory text field but can actually accept a complex JSON object with many "blocks" and create a high level of interactivity. On top of this, two of these protocols need a recipient to be explicitly identified (SMS and Email) and the other does not.
How do we set up a UI that knows something about these protocols, on top of a rendering system that does not? Or do we need to make the templates protocol-specific? Or do something simple like, have each notification template be a map of string → Mustache template, and the UI just shows N templates per notification template?
4 Comments
Rick Lively [X]
I think this is generally what TTA Tools would want. Using a templating mechanism similar to the PHT disposition letters would be needed. But I could also see using for other less complex messages such as proposal submission, review completion etc. It would be nice to have configurable groups of recipients so individdual emails wouldn't alway be needed. For example retular proposal submission (TTA group), DDT submission (TTA group, telescope operations), sponsored proposal submission (sponsor contact). Having an option for email, SMS, Slack would also be nice as, in general, most people will use email, but I could also see wanting SMS notification if a hot DDT proposal gets submitted.
Daniel Lyons
My thinking is that the notification system doesn't know anything specific about recipients, it just expects that when the rendering step is complete it can hand off to a protocol for it to be sent. So let's pretend a TTA user submits a proposal and we want to send them an email thanking them for their submission and notify their science category reviewers that a proposal has arrived for them. This isn't probably what you'd want IRL but it's to illustrate sending to a group and a user.
The TTA system would trigger the new-proposal notification. The payload would include the proposal ID, title, user's email, user's formal name, the science category name and the science category reviewer's email addresses.
The notification system would look up the templates for 'new-proposal' and find two templates. The first one is the "thanks for your proposal" email template, and it fills in the proposal ID, title, and user's email. The recipient is the user's email. The second template is the "new proposal for {science category}" it fills in the proposal ID, title, user's name, and the recipient is the list of science category reviewer emails, in BCC. Two templates, two emails (effectively more, because of the BCC mechanism).
If we wanted to send an SMS for DDTs, we would make a second notification 'new-ddt-proposal' which would have those two templates and a third for the SMS.
The TTA application winds up being responsible for gathering up all the information that the notification could need, but it just dumps it on the notification service as a JSON blob, and the rest of the work happens there. Code changes are needed only for introducing new notifications or adding new information to the notification call.
Jeff Kern
I like where this is going. I think there is an important design choice here between putting the onus for multiple notifications on the service or on the application. I'd say that what you propose above Daniel put this on the service which means when we want to change a notification (which is a configuration property per solicitation for TTA) we have to somehow modify the service. I would leave the logic for replication etc to the application and have the notification service be fairly simple. In my mind the notification service would get: template-id, json, list of users. The service is responsible for instantiating the template using the json and then sending the notification to all users in the list.
I think there are two and possibly three types of users:
Daniel Lyons
You don't modify the service, the service has multiple message templates per notification. You go in through the notification service's UI and construct new templates to send multiple notifications. This isn't going to take 100% of the onus off the application, because the application may still need to send different payloads, or have different recipients. If the recipients of a notification are just another argument, for instance, then it would be easy to do something silly like hard-code a recipient for an SMS message. This is the "every time a DDT is submitted, send Dana a text" concept. That would be a single notification, as far as TTAT is concerned, and changing it to do or not do that would be dinking around in the notification service UI. The question "do I need to modify the application?" would have the same answer as "does the notification service get enough information from the application to send this notification?" Let's call this the two-argument regime, because here the notification service gets two arguments: a notification name and a data payload.
Let's suppose the recipients are extracted and sent as a third parameter. I'll call this the three-argument regime. We need some support here from A3, because we need to know what the right message destinations are for the user and whether they are opting into this protocol or not (providing your phone number is not equivalent to consenting to receive SMS messages). The "send Dana a text" notification now has to be sent separately by the TTAT application because Dana is not the same recipient as the user. TTAT would also have to send a separate notification for the group notifications. We can still have an SMS message template and an email message template, and users who want one or the other or both could be serviced by a single notification as far as TTAT is concerned. Basically, the question "do I need to modify the application?" would have the same answer as "does this notification have different recipients?"
The user types breakdown you give shows the benefit of having A3 there to handle notification preferences, but it also shows that both of these approaches are going to run into some issues dealing with anonymous users and it isn't clear how A3 helps us with those. I can say I don't really think I would enjoy moving conditional logic into the templates to address different classes of recipient (A3-identified and address-only). If I wanted recipient classes (and protocol preferences) to be handled by an earlier step than template rendering by the notification system, it would probably have to call them out separately, which means we're in the three-argument regime.
I like the flexibility of the two-argument regime a lot, but I think the clarity of understanding when the application needs to be modified might be beneficial even if it is less flexible, and the A3-integration may make it easier to handle the known/unknown user classes.