How Work Actually Runs

You press Finalise on a cash bill and the screen comes back almost at once. Nothing looks different. Yet within the next few seconds a journal is written, stock leaves a location, the item’s moving-average cost is recalculated, a knock-off queue opens for the delivery order that comes next, and an e-invoice row appears in a pool.

None of that is on your screen, and none of it happened while you were waiting. This page is the part underneath: the job processors, which are how almost everything in BigLedger that is not a keystroke gets done. Read it once and a dozen otherwise baffling behaviours stop being baffling — why a document changes after you saved it, why an error arrives ten minutes late and lands on nobody, why a colleague’s tenant does something yours does not, and why “it hasn’t posted yet” and “it will never post” look identical on the screen.

Twenty minutes. Nothing to configure; this is orientation, not a procedure.

A processor is not one thing — it plays four roles

Vincent’s own description, and the reason this page exists rather than a list of which processors to switch on:

“A processor is used in a variety of ways in our platform. They can act as queue / publish-subscribe / actors / schedulers… There’s primary job processors that can trigger secondary job processors, it can run synchronously or asynchronously, it can also have cron schedulers that will clear up what’s supposed to run, but didn’t run, from the job processor queue.”

Four roles, one mechanism. Here is each one with a real example, and one honest caveat.

As a queue — work waits in a table, and a worker claims it

Every job is a row in a plain database table. On your tenant that table is app_scheduler_queue: an event code (which processor), an event-properties blob (usually just the document’s id), a start time, an expiry and a priority.

A worker takes a job by deleting it:

DELETE FROM app_scheduler_queue WHERE ctid = (
  SELECT ctid FROM app_scheduler_queue WHERE event_code = :eventCode
   AND start_time <= NOW() AND expire_time >= NOW()
   ORDER BY priority, start_time
   LIMIT 1 FOR UPDATE SKIP LOCKED
) RETURNING ROW_TO_JSON(app_scheduler_queue) AS data

FOR UPDATE SKIP LOCKED is the whole horizontal-scaling story in one clause: several workers can read the same queue at the same time and each simply steps over rows another worker has already locked. Add capacity and the same code goes faster. Every tenant has its own database, so the work shards by customer for free.

The consequence you have to hold on to is that claiming a job destroys the evidence that it was queued. There is no “in progress” row and no “done” row. A job is either still waiting or gone.

As publish-subscribe — and this one is literal, not a metaphor

When a document is finalised, the primary processor does not call the journal, then the stock, then the points. It publishes, and whoever subscribed gets a job.

The subscriptions live in two places, which is the single most useful fact on this page:

A secondary only runs if your tenant holds a row for that subscription. So the answer to “does BigLedger post a journal when I finalise an invoice?” is genuinely it depends on your tenant — and that is a feature, not a gap. A group with no loyalty scheme does not run the points processor. A company that keeps its accounts elsewhere does not run the journal processor. Measured across all 90 tenants on 2026-09-17, aggregates only:

Those are not failures. Many tenants in the fleet do not keep stock in BigLedger, or do not keep books in it. But it does mean no page anywhere — including this one — can tell you what your finalise does. Only your tenant’s subscriptions can, and there is no screen that shows them today. That is a real gap and we have raised it; in the meantime it is a question for BigLedger.

On top of the subscription there are two more filters, and both are worth knowing because they explain “it posts for sales invoices but not for cash bills”:

• A document-type constraint on the subscription itself. A link can name the server_doc_type values it wants; a document of any other type is skipped silently.
• A per-company include/exclude list held on the company record. If it names an include list, only those processors run; anything on the exclude list never runs for that company.

As a scheduler — cron, per tenant, with a run log

Some work is not caused by anything you did. It has to happen at ten past the hour, or at 1 a.m. on the first of the month. That is bl_sch_crontab_hdr: one row per scheduled job on your tenant, with a standard cron expression, a status, an execution strategy and a last_run_time.

All 90 tenants have some. The median tenant has 11 active clocks; the busiest has 55. Seventeen of them are defaults that BigLedger inserts when a tenant is created — the e-invoice submission and status-update passes every ten minutes, the consolidation pass every twenty, the stock-balance queue every fifteen, the deposit rollover and the monthly aging on the first of the month.

The words when a tenant is created are doing a lot of work in that sentence. The sync only inserts a clock that is not already there, and it runs for new tenants — so a job added to the defaults after your tenant was created does not appear on your tenant. That is the single commonest reason a feature that “ships” does nothing for you.

As an actor — the one place the code does not match the description

This is the role we could not confirm as described, so we are saying so rather than forcing the fit. The platform’s own vocabulary for a processor has exactly three values — PRIMARY, SECONDARY and SYSTEM — and there is no actor framework: no mailbox per entity, no single-threaded owner of a piece of state, no supervision tree.

What is there is exclusivity where it matters, done with named locks rather than actors:

• bl_job_processor_lock — a named lock row with an expiry, taken by a job that must not run twice at once (the document ETL path, the import-failure mailer). The expired ones are swept on every pacemaker pass. Across the fleet it holds 14 rows in 14 tenants at any moment — it is a live latch, not a log.
• A per company and item lock taken by the costing pass, so two documents touching the same item cannot both recalculate its FIFO chain at once.

So the behaviour Vincent describes — one owner, serialised work, no two runs colliding — is real and is what the locks buy. The mechanism is not an actor system. Flagged for Vincent: if “actors” points at something else in the platform (a WMS device handler, the chat queues) we have not found it, and this section should be rewritten rather than left as our inference.

Trace it once: what one Finalise actually starts

Finalise is its own endpoint, not a save. If you go looking for the posting in the code or the audit trail of an ordinary update, you will never find it, because it is not there. This is the most useful sentence in this page for anyone who has tried.

Here is the whole chain for one finalised document, end to end.

    flowchart TD
  F["You press Finalise<br/>(a separate endpoint, not Save)"]
  Q["One row inserted into app_scheduler_queue<br/>BLG_ERP_GENERIC_DOCUMENT_PRIMARY_PROCESSOR"]
  R["HTTP response returns here —<br/>the chain runs on a background thread"]
  P["Primary processor: look up this tenant's subscribers"]
  J["Journal posting"]
  I["Inventory transaction lines"]
  K["Knock-off open queue"]
  E["E-invoice pool / matching"]
  M["Membership points"]
  O["…26 others, per subscription"]
  B["Stock balance per location and company"]
  C["FIFO / LIFO costing — cost written BACK<br/>onto the document line"]
  F --> Q --> R
  Q --> P
  P --> J
  P --> I
  P --> K
  P --> E
  P --> M
  P --> O
  I --> B --> C
  

Step by step, with the names:

1. Finalise inserts one queue row for BLG_ERP_GENERIC_DOCUMENT_PRIMARY_PROCESSOR and then hands the work to a background thread pool. Your HTTP response returns without waiting.

2. The primary processor asks who subscribed. In the master registry, 32 secondary processors are linked to this one publisher. Your tenant’s own subscription rows decide which of the 32 are eligible, then the document-type constraint and the company include/exclude list narrow it further.

3. Each surviving secondary gets its own queue row. Thirty-one of the 32 links are marked RUN_NOW; one is INSERT_TO_QUEUE.

4. A secondary can itself be a publisher. This is where the chain gets deeper than two levels, and where the costing write-back in the ripple map actually happens:

   INVENTORY_TRANSACTION_LINE_PROCESSOR → INVENTORY_TXN_LINE_TO_CURRENT_STOCK_BALANCE_PROCESSOR → INVENTORY_FIFO_LIFO_COSTING_PROCESSOR

   The last of those computes the cost and writes it back onto the document line you finalised. Your document is not finished when you finalise it. It is finished when the chain has run — and it is a loop, not a line.

Seven other publishers have their own fan-outs in the same registry, and the shapes rhyme: voiding a document publishes to nine void processors, discarding publishes to three, a cash document publishes to a cashbook, a journal and a post-dated-cheque processor.

Synchronous or asynchronous — who pays the wait

There are exactly two execution strategies, and the difference is whose thread does the work.

The trick that makes finalise fast is that the two are combined. Finalise uses RUN_NOW — but it first pushes the whole thing onto a background thread pool (30 threads, fixed, per API instance). So:

• Your cashier never waits for the journal. The till’s Finalise returns as soon as the queue row is written. A twenty-second call to a regulator never shows up as a twenty-second checkout. You can verify this at your own till in thirty seconds.
• The background thread pays for all of it, serially. Because the secondaries are RUN_NOW, the primary’s thread runs subscriber after subscriber in turn. Thirty-two subscribers on one thread means the last one starts after the first thirty-one have finished.
• Thirty threads is the ceiling per instance. A burst of finalises beyond that queues behind them — which is fine, because the queue is exactly where the work is safe.

This is also why errors do not land on the person who caused them. By the time the journal processor fails, the request that finalised the document returned long ago, and there is no thread to raise the error on. Nothing is shown at the till, nothing is e-mailed, and the person who pressed the button sees a perfectly normal document.

The cron sweeper: exactly what it catches, and what it does not

This is the part that makes asynchronous work defensible, and it is also the part our own pages have most often got wrong in both directions.

What drives everything

A pacemaker tick arrives from outside the application — an AWS CloudWatch/SNS message, repeated if it is not processed within a minute. It runs MAIN_GLOBAL_PROCESSOR, which loops every active tenant and puts a MAIN_TENANT_PROCESSOR job on that tenant’s queue. That per-tenant processor then does three separate things on every pass:

1. Fires due clocks. It reads your active bl_sch_crontab_hdr rows, works out which are due, queues each one, stamps last_run_time and writes a row into bl_sch_crontab_event.
2. Drains the queue. It selects up to ten due event codes sitting in app_scheduler_queue and drives each one. This is the sweeper in the plainest sense: anything that was queued and never ran — because a process restarted, a thread died, the instance was replaced mid-chain — is still a row, and the next pass picks it up.
3. Takes out the rubbish. It deletes expired queue rows, rows whose retries are exhausted, and expired job-processor locks.

So the honest general answer is: work that is still in the queue always gets run eventually. Work that was claimed and then failed does not.

The bit that surprises people

Remember that claiming a job deletes it. On the main tenant queue the handler then runs outside a transaction, and if it throws, the code writes an error row and stops. The queue row is not put back. Nothing retries it.

(Other queue families in the platform behave better: they roll back, decrement a retry counter and try again until it reaches zero. The main tenant queue — the one your document postings go through — is not one of them.)

So “the sweeper will get it” is true for a job that never started and false for a job that started and blew up. Those two cases look identical from your desk.

What closes that gap, where anything does: the watchdogs

A watchdog is a different idea from the sweeper. It does not look at the queue at all — it looks at your documents, finds the ones that are missing a posting, and queues the job again from scratch. That is the piece that genuinely answers “nothing tells you it ran”.

The inventory watchdog exists and works. Every run it finds every document where posting_status = 'FINAL' and the inventory posting status is still blank and the document is more than twenty minutes old, and re-queues the stock posting for it — skipping anything already queued, so it cannot pile up duplicates. It does three more jobs besides: re-posts stock reversals for voided documents whose stock never came back, clears draft serial-number locks left behind by documents that were finalised or discarded, and queues a refresh wherever the stock ledger and the stored balance disagree.

It is registered, and it is scheduled on 20 of 90 tenants. If your tenant is one of them, a sales invoice whose stock posting failed will be picked up and retried within the hour without anybody noticing. If it is not, it will not.

There is no journal watchdog. Nothing anywhere re-selects documents whose journal is missing. And that matters more than it sounds, because the journal posting job is the most-failed job in the entire fleet: 2.2 million error rows across 51 tenants, against 1.6 million for inventory posting across 33.

The knock-off watchdog is written but cannot run. The code exists and does exactly the right thing — find finalised documents whose knock-off queue was never opened, re-post them. But its queue code is not in the registry the scheduler resolves names against, and the scheduler drops an unrecognised code silently. Even if somebody added a clock for it, nothing would happen and nothing would say so.

So: the sweeper covers the queue. One watchdog covers stock, on about a fifth of tenants. Nothing covers the ledger.

Shipped, registered, scheduled, running — four different facts

We got this wrong twice on our own pages in one day, so it is worth being blunt. A processor can be in any of four states and only the last one means anything to you.

Two live examples, both measured on 2026-09-17:

• E-invoice ghost-document detection — the standing check for finalised, e-invoice-eligible documents that never reached any e-invoice pool. It shipped. It is registered. It is in state 3’s waiting room: it is not one of the seventeen default clocks, and zero of 90 tenants have a clock for it. So no digest e-mail will ever arrive, on anybody’s tenant, and a page that says “standing detection now runs” sends you to wait for something that cannot happen.
• The knock-off watchdog — stuck at state 2. Not registered, therefore unrunnable, therefore also zero of 90.

State 4 is the one with good news in it. Every clock that fires stamps its own last_run_time and writes a run-log row, and across the fleet that log holds about 35 million rows on all 90 tenants. It is the answer to “did this ever run” — and of the roughly 1,100 active clocks in the fleet, 930 last ran within seven days and 160 have not run in over a week. Whether a given stale clock matters depends entirely on its cron expression, so that number is a prompt to look, not a fault count.

How you know it worked

Three checks, in the order of how much work they are.

One document, thirty seconds — the Posting tab. Open a finalised document and look at its Posting tab. It shows five statuses read straight off the document: Journal, Inventory, Membership Points, Cashbook and Tax. About twenty document applets carry this tab, including sales invoice, purchase invoice, sales return, receipt voucher, delivery order and consignment billing.

Read it like this:

• POSTED — that subsystem is done.
• Blank on a FINAL document — the job has not run or it failed. Those are the same value. There is no third value here: the journal processor writes POSTED on success and never writes FAILED, so a failure looks exactly like a job that has not got there yet.
• Give it a minute before you conclude anything. If it is still blank after twenty minutes on a tenant with the inventory watchdog, the stock posting has been retried at least once already.

One document, properly — Trace Document. In the Financial Report applet, Error Checking → Trace Document takes a document type and number and runs six checks against it: journal exists, journal is right, cashbook line exists, shadow document exists, and for a voided document the reversal and its queue. Each failed check carries a Resolve button that creates or re-posts the missing piece. Use Resolve rather than trying to finalise again. Resolve is the path the product built for this; re-running Finalise on a document that is already FINAL is not a repair, and our Chart of Account applet page records that a second finalise is refused. We were not able to pin that refusal in the backend ourselves, so treat the refusal as reported rather than verified — but Resolve is the right move either way.

In bulk — the Stock Flow Report. Inventory value against accounting value, per module, with a difference column you can post from. This is the one to put on a routine, because nothing will prompt you.

When it goes wrong

What this will not do

Stated plainly, because a limit saves more time than a feature.

• It will not tell you a job failed. There is no alert, no e-mail, no badge and no entry in any screen you can open. Failures go into a table (app_scheduler_std_error) that holds roughly 9.8 million rows across 87 of 90 tenants and that nothing in the product reads — not a report, not a screen, not a digest. Sixteen thousand of those rows were written in the last twenty-four hours. We have raised it as a product issue. Until it is fixed, a failed job today tells nobody, and your own routine is the only cover.
• It will not retry a posting that failed. On the queue your documents go through, a claimed job that throws is gone. Only the inventory watchdog re-creates work from the documents themselves, and only on the tenants where it is scheduled.
• It will not show you which processors your tenant runs. There is no applet for subscriptions and none for clocks. This is the honest limit behind every “ask your BigLedger contact” you will read on our pages, and it is a real gap rather than a documentation failure.
• It will not put a failed document right by finalising it again. Resolve the check in Trace Document instead.
• It will not order the secondaries for you beyond what the chain fixes. Costing runs after stock because it is a child of the stock job, not because anything sequences the fan-out. Two subscribers of the same publisher have no guaranteed order between them.
• It cannot run a processor whose code is not registered, and it will not tell you that it skipped one.
• The failure count is not a document count. One row is one failed attempt, and some queue families write a row per retry, so 9.8 million rows is an upper bound on the damage, not a measure of it.
• Queues can fall behind badly and silently. One tenant in the fleet is holding about 910,000 inventory-posting jobs queued more than thirty days ago. The queue is doing its job — the work is safe and will run — but nothing announces a backlog of that size to anyone.

Where this sits in the product

What has to exist first. A subscription on your tenant for the subsystem you care about, and — for the journal — a default GL code the posting can resolve. Without the GL mapping the job finalises the document and then fails, and you find out at month-end. Without the subscription nothing is even attempted.

What this makes possible. Everything the ripple of one finalised document buys you: nine consequences from one keystroke, each of which is a person and a spreadsheet in a business running separate systems. It is also what lets the till stay cheap and uninterruptible while everything expensive, fallible or regulated happens on the reporting side.

Which is which — sweeper against watchdog. They are not alternatives and people mix them up. The sweeper drives work that is still in the queue; use it to explain why something ran a minute late. A watchdog re-creates work from your documents when the queue no longer has it; use it to explain why a stock posting recovered by itself. If you are asking “will this fix itself?”, the question is which of the two applies — and for the ledger, neither does.

What breaks if this is wrong. A subscription that is off does not fail loudly; it fails invisibly, across every module at once. Journal off means no ledger and a clean-looking document. Inventory off means stock balances that quietly diverge from the movements behind them. Knock-off off means every downstream applet’s Search Document tab comes up empty and the reason is on a company settings screen nobody thought to open.

What it pairs with in practice. The document itself — see The Generic Document for why a cash bill and a purchase invoice behave identically here — and the reporting snapshots, because a job that has not run and a snapshot that has not been rebuilt produce the same complaint from the same person. That is Why Two Screens Show Different Numbers.

Related documentation

• The Generic Document — the record all of this runs on, and why one explanation covers 97 document types
• Why Two Screens Show Different Numbers — stored snapshot against live recomputation; the other half of “why is this number wrong”
• Stock Balance — Background jobs — the two jobs that take a finalised document into the balance, at the applet’s own level of detail
• Financial Report — Error Checking — Trace Document and the Stock Flow Report, the two screens that answer “did it post”
• Organization applet — knock-off pairs and the company settings the chain reads at FINAL
• Understanding e-Invoice statuses — the same machinery seen from the one subsystem where the statuses are visible to you