Scheduled Jobs
Scheduled jobs have three components:
jobs- the top-level operation that defines when a job is executed by calling aserviceservices- the code for the jobscheduler- the long-running process that executes jobs according to their scheduling configuration
Jobs
The jobs.py file is intended as a shorthand index of available jobs.
Each top-level job in jobs.py should contain as little code as possible,
ideally just calling a single service function.
Each job should have its own configuration in the config object
that allows it to be enabled/disabled and otherwise have its scheduling configured.
Services
Services should be relatively self-contained, such that disabling a service does not cause the site to become unusable.
Services should be async and make careful use of resources, such as limiting
the number of concurrent async calls with a Semaphore or otherwise.
Avoid long-running processes, and split jobs into small, atomic units when possible
so that failure of any part of them doesn't break the whole thing.
Services should not have side effects except for mutating the database.
Services should export a single function with no parameters up to sciop.services to be called from jobs.py
Scheduler
The scheduler module wraps APScheduler with a handful of convenience methods. It is started and torn down with starlette's lifespan contextmanager methods.
The scheduler uses the database as its job store, so jobs can be persisted in the case of server crashes, though currently they are removed when the server is requested to do an orderly shutdown.