Background Jobs

Background jobs as the name suggests are jobs that run in the background independantly of the main execution flow of the system and run without user interaction. These can be used for variety of purposes:

• CPU intensive jobs
• IO intensive jobs
• Batch jobs such as scheduled processing
• Long running workflows
• Sensitive jobs in which we send jobs to much more secure location for processing.

Triggers

You can initiate background jobs in several ways. They fall into one of the following categories

• Event driven triggers: Event driven triggers uses a specific action/condition that must be fullfilled in order to initiate the event driven background job. There are various types of triggers such as user clicking a button, incoming http request to the server, database trigger to automatically update the timestamp of a row when data changes, etc
• Schedule driven triggers: Schedule driven triggers are those job that are initiated on a schedule such as cron jobs, job to backup the database, etc

Returning results

Since background jobs are created to be independant of the main execution flow, the calling process doesnt wait for background job to finish and cant automatically detect when the job ends. To create the background job communicate with the main process we can do the following:

• Return a status endpoint to the caller
• Use a reply queue
• Push notifications through events
• Call back to the caller using a webhook
• Write to shared memory