Submitter
[Needs Reviewing]
Submitting user data
Once a user’s data has been collected and the user has confirmed that the data is correct, urls representing the required output of the data and any uploaded files are sent to the submitter api
Submitting overview
[to add] Submission overview diagram
Default output
Send copy to user
In order to send a copy to the user, there must be an email address to send to:
- If a value has been defined in the service configuration for
emailInputNameUser, the value entered for the control with that name is used as the user’s email address - Otherwise, if a control named
emailexists, the value for that is used
Outputs
- Email message
/api/submitter/email/user
- PDF output
/api/submitter/pdf/default/:submissionId.pdf
Send application to service team via email
In order to send the data to the service team, there must be an email address to send to. This address is set through the Publisher as a config param SERVICE_OUTPUT_EMAIL
Outputs
- Email message
/api/submitter/email/team
- PDF output
/api/submitter/pdf/default/:submissionId.pdf
- CSV (TODO)
JSON output
It is possible to get raw JSON for a submission from the Runner application. Currently this is forwarded onto adapters to make external
API calls. In order to use this, two config params are required. SERVICE_OUTPUT_JSON_ENDPOINT and SERVICE_OUTPUT_JSON_KEY.
SERVICE_OUTPUT_JSON_ENDPOINT
The endpoint to which this JSON will be forwarded.
SERVICE_OUTPUT_JSON_KEY
A JWE key used for encrypting the data. This key has to be exactly 16 bytes.
- API endpoint
/api/submitter/json/default/:submissionId.json
Output specification
In future, it will be possible to define alternative outputs
TODO: output specification
Repo: fb-submitter
The Submitter is an app that provides an API and workers for dealing with a form’s submissions.
Submitter API
Introduction
When a user has completed their journey on a FormBuilder service, they will be presented with a ‘Check your answers’ page with the option to submit their data.
Once they click to submit, they should immediately get a confirmation message, with a reference number that they can use for future reference if needed. The component that generates this reference number and takes responsibility for sending the data to where it needs to go, is the “Submitter”.
This document summarises requirements for the Submitter, its API calls, parameters & responses.
System Context
TODO: Insert image
API
Constraints: No Ingress, JWT authentication, JSON-only, HTTPS-only (NB. dependent on Cloud Platforms providing a solution for cluster-internal SSL)
Following the same pattern as the User Data Store, each request will be timestamped and signed using a per-service serviceToken (generated by Publisher and injected into the service’s Runner as an environment variable).
Methods
Every request must be signed using JWT with the service’s serviceToken (this will be injected into the Runner’s container as an environment variable called SERVICE_TOKEN, by Publisher) Signing options should include iat: (current timestamp)to allow a time-limit and mitigate against replay attacks
The resulting token must be passed as a header called x-access-token
POST /submission
When the user clicks the final button to submit their data, the service should should perform a POST request to /submission, passing a JSON packet in the body of the following form:
{
"service_slug": "...",
"encrypted_user_id_and_token": "(userId and userToken encrypted via AES-256 with the serviceToken as the key - sent back to runner as ‘x-encrypted-user-id-and-token’ header)",
"submission_details": [
{
"type": "email",
"to": "email address of recipient",
"from": "email address replies go to",
"subject": "(mail subject line)",
"body_parts": {
"text/html": "(relative url)",
"text/plain": "(relative url)"
},
"attachments": [
"(relative url 1)",
"(relative url 2)",
"(...)"
]
},
["..."]
]
}
- If the request was successful:
- A submission record will be created to store the given details
- A job will be placed on the queue for asynchronous execution
- The response status will be 201 Created
- The body of the response will contain a JSON packet of the form:
json { "id": "(uuid of the submission for future reference)", "created_at": "(timestamp in ISO-8601 format)", "updated_at": "(timestamp in ISO-8601 format)", "status": "(queued|processing|retrying|completed|failed)" }
- If the token is not present, the response will be 401 Unauthorised
- If the token is not valid, the response will be 403 Forbidden
- Other failure types will result in the corresponding standard HTTP response codes (400, 500, etc)
GET /submission/:submission_id
Allows retrieval of the submission status. Returns a response of the form:
{
"id": "(uuid of the submission for future reference)",
"created_at": "(timestamp in ISO-8601 format)",
"updated_at": "(timestamp in ISO-8601 format)",
"status": "(queued|processing|completed|failed)"
}
Submitter Workers
In the same manner as Publisher, there will be separate containers/pods for the back-end worker processes, which will be responsible for retrieving the PDFs and actually sending the emails via Amazon SES. The queue will be Resque using Elasticache Redis, and the job framework will be ActiveJob with a Resque backend.
When processing a submission, the job will:
- Update the submission record status to
processing - Read the email submission details
- Retrieve PDFs from all of the given URLs (in the event of the same PDF being required by multiple emails, it will only retrieve the PDF once)
- Construct an email to each recipient address, attaching the relevant PDFs
- Send the email via Amazon SES
- Update the submission record status to
completed - If there is a retryable error (e.g. timeout retrieving URLs) the job will automatically retry, and the submission record status will be updated to
failed_retryable - If there is a non-retryable error at any point, the submission record status will be updated to
failed_non_retryable