Deployment

Release

Create a release with lein release in mdr2. This tags the version and triggers the release github action which uploads the uberjar to the release page. Then deploy it with ansible as described in the sbs-infrastructure README.

Note that the madras2 role depends on the pipeline1 role, so a deploy also installs whatever pipeline1_version is set in sbs-infrastructure.

Restarting safely

The work queues (encode, archive, repair and the notifications to ABACUS) are in-memory channels. A deploy restarts the server and anything still in a queue is lost. The state of a production tells you whether there is pending work:

Queue	State of a pending production	Effect of a lost entry
encode	recorded or split	production is stuck in that state
archive	cataloged	production is stuck in that state
repair	archived (repairing once started)	repair silently doesn’t happen
notify-abacus	any state change	ABACUS has a stale state until the next change or status request

Before the deploy

Pick a quiet time. Recorded productions trigger the expensive encoding. According to the workflow they are pushed from the ABACUS machine after office hours (periodicals immediately), so a weekday morning is usually best. The actual timing is defined by the cron job on the ABACUS machine, not in mdr2.

Make sure nothing is pending. The API lists productions by state without a login:

for s in recorded split cataloged repairing; do
  echo "$s: $(curl -s "http://madras2/api/productions?state=$s")"
done

Every state should return []. Repairs that are queued but not yet started cannot be seen this way, as the production stays archived.

If this returns rows, wait until they are processed. Ask the studio not to start repairs during the deploy.

After the deploy

Run the check again. If a production got stuck, re-queue it via the nREPL (port 7000 on the server):

(require '[clojure.core.async :refer [>!!]]
         '[mdr2.queues :as q]
         '[mdr2.production :as prod])
(>!! q/encode {:production (prod/get-production 1234)}) ; stuck in recorded
(>!! q/archive (prod/get-production 1234))              ; stuck in cataloged

If an archive job was interrupted half way, check the archive spool directory and the archive db before re-queuing it.