Explanation from Edplay CTO, Myroslav

Most AI training-content tools generate an outline from a topic and then write pages, and the teaching quality depends on what the model happens to produce. We add an instructional design phase before any writing, and the later steps are not allowed to change it. The order:

• Brief. A fixed questionnaire, not an open chat: audience, setting, prior knowledge, goals, tone, size, teaching style. Uploaded documents are read here. The customer chooses how closely to follow them: Mirror (the document's own headings become the structure), Expand (stay close to the documents, explain and illustrate but add no facts) or Reimagine (new training content from their substance, every claim still cited and checked).
• Plan. A planner model writes a typed data structure, not prose: measurable objectives tagged by cognitive level (Bloom's), prerequisite order, the misconceptions learners usually bring, an assessment strategy, and a section layout built only from the blocks the chosen learning mode allows. Six learning modes ship built in (teach a topic, train a process, practice decisions, support performance, refresh knowledge, test competence) and customers can define their own. Each mode is nested data: phases contain sections, sections contain teaching blocks, each block names the component types allowed inside it, and each component type is a strict schema. The teaching method lives in that configuration, which we and our customers can edit, not in prompts.
• Plan checks, in code. Eight teaching rules run deterministically: every section has a goal, basics come before the things that depend on them, the common misconceptions are addressed, practice moves from guided to solo, there is recall practice, there is a test, the test matches the style's rules, the page count fits. Entries outside the allowed list are dropped or repaired before any model is paid, and unsound plans are flagged; automatic send-back of a weak plan is designed and not yet switched on.
• One running story. A single case (one workplace, one person, one problem) is generated before structure and carried through every section, so all of the training content follows one narrative. A second model turns the same setting into art direction, so images show the same place and tools as the text.
• Structure. Transcribed from the approved plan, never redesigned. Sections, then pages, each with a goal, a purpose, and a one-line brief per component. Still no content.
• Review, then a person. Code checks that every planned component exists and is allowed, that a practice block has something to do, and that a test block really tests. A reviewer model then grades each section against its own plan and must quote the exact text behind every problem it raises; our code computes pass or fail from the quotes, not from the reviewer's overall opinion. Weak sections are redone, up to twice. Then a human approves or edits the structure. This gate is mandatory: nothing is written before a person signs off. A second gate on the plan itself can be switched on per workspace.
• Content, all pages at once. One job per page, in parallel, across more than 50 specialised agents. Each writer receives its own slice of the plan, a map of what the neighbouring pages cover with the rule "build on it, do not repeat it", the running case at this section's stage, the retrieved source passages, the organisation's approved definitions, and the exact format of every component (50 asset types, each a strict schema). It works through functions our code provides (get a passage, make a component, make an image, search the web, find a video) and never invents a name or an ID; our code assigns every identifier. As each sentence is written, we record which passage it came from.
• After writing. A vision model checks every image against what the page needed. Every citation is re-checked: does the passage really support the sentence? Sentences without a source are counted and shown. Every piece of training content, page, component and quiz question is linked to the skills it teaches or tests, so a learner's first answers count toward mastery. The training content is indexed so search and the learner-facing tutor can find its text.

A typical build takes about seven minutes end to end, through two human gates and every check above.

We treat every model output as a draft that has to be verified. Cheap deterministic checks run first, model-based review second, and humans decide at the gates that matter. The main mechanisms:

• Naming and identifiers. The model is never allowed to name things. Every concept, plan unit and component gets an identifier assigned by our code, and all references resolve through those identifiers. This is what makes the plan enforceable instead of decorative.
• Parallel writing without duplication. Pages are written in parallel, which would normally mean many workers explaining the same idea twice. Instead of coordinating them, we partition: the plan's concepts are divided into exclusive page-level territories before writing starts, and each page carries its own slice plus a record of what its neighbours cover, resolved from the live, human-approved training content rather than a snapshot.
• Review with evidence. Reviewer models return structured findings with mandatory quotes; a finding without a quote is discarded. Pass or fail is computed by our code from that evidence, never taken from the reviewer's summary, so a weak section cannot pass on a general positive impression. Content review samples the most cognitively ambitious page of each section first and only reads the whole section if that page fails.
• Targeted repair. Each confirmed defect traces to the one component whose text was quoted. We delete that component and rewrite it in place with full teaching context, instead of regenerating the whole page. Known model quirks (JSON inside a string, a nested single-item list) are normalised before checking, so retries are spent on real errors.
• Image validation, before and after. The image prompt is built from the asset and its teaching role, then reviewed against six criteria before any image is generated. After generation, a vision model judges the pixels against what the page needed. The verdict and the prompt are stored on the asset, and a failing image is edited in place or regenerated with a revised prompt.
• Bounded rounds, no dead ends. Repair rounds are capped (two for structure, a set number for content), retries are budgeted by wall-clock time rather than attempt count, providers fail over per call, and anything unresolved is listed for a person. A build always finishes and always states what was not fixed.

Underneath, a build is a long-lived distributed transaction: an 8-node workflow that parks on persisted interrupts (a human gate, a pending batch) and resumes over server-sent events, so it survives restarts, deploys, and a reviewer who goes to lunch at the approval gate. Page jobs release before the hard kill signal and resume by comparing planned versus present, regenerating only the missing assets; crash recovery and review repair share one code path, so the recovery path is exercised daily. A Redis-backed semaphore caps in-flight model calls across every pod and every build, credits are deducted exactly once per page under idempotency keys, and a per-page mutex prevents overlapping executions of the same work.

Customers upload SOPs, manuals and policies. The Workspace Brain turns them into structured understanding: a concept inventory of up to 200 ideas with definitions and up to 100 "you need A before B" links, procedures as ordered, editable steps, suggested glossary terms and skills, and a record of where each appears in each document.

Near-duplicate concepts are found by embedding similarity, but a model makes the merge-or-new decision, so "authentication" and "authorization" stay distinct. Every piece of training content is then held accountable to this understanding: the planner must teach every idea in the inventory, keep the prerequisite order, and add ideas only as bridges the learner would need.

• Citations, including missing ones. As each sentence is written, we record which passage it came from. An automated audit re-checks every link and marks it verified, weak or rejected; a second pass looks for sources for sentences that have none, and what remains unsourced is counted and shown per component. That count answers the question buyers ask: what did the model add on its own?
• House vocabulary, enforced at four points. The skills catalog holds the organisation's approved terms and skills, each with an official name and definition. It is seeded from quiz questions and document concepts, never from generated pages, so the generator cannot write the ruler it is measured with. Before writing, the ten most relevant approved skills are handed to the writer with their definitions. At save time, up to eight relevant terms are checked; a component that contradicts a definition, or teaches a term under another name, is rejected with an exact quote and the official definition, and rewritten. During content review, the reviewer sees the same definitions. And every piece of content is linked to skills afterwards: the linking model may only choose from candidates that search found, a link with confidence 0.85 or higher is approved automatically, and anything lower waits in a human review queue. Only approved links count toward coverage.
• Coverage, measured. For document-based training content we compute which of the document's topics each page teaches and what was left out, with the denominator taken from the document itself.

When a document is revised

The parts exist; the algorithm that joins them is in progress. On upload, the uploader names which file the new one replaces (the form suggests a match when a file with the same name and different content exists; the identical file is refused as a duplicate). The old file is marked superseded only when the new one finishes processing, so a failed upload never demotes a working source. Comparison then runs in code, with no model: sections are matched by path and heading similarity, and passages are compared within matched sections. At 0.97 similarity or more a passage is unchanged, between 0.60 and 0.97 it changed, below that it was removed. The recorded citations then name the exact affected pieces, and each gets one of four fates: rebind (unchanged passage, the citation is re-pointed, nothing rewritten), refresh (changed passage, the component is rewritten), orphan (passage gone, a person chooses keep, delete or rewrite), gap (a new section nobody teaches, listed for the admin). The admin approves item by item with the old and new passage side by side; nothing changes for learners before approval. After approval, only the cited pieces are rewritten, their citations re-checked, and a refresher goes to exactly the learners who had proven the old rule.

Example: version 2 says returns are accepted within 14 days; version 3 says 30. Two components across two pieces of training content cite that paragraph and one quiz question tests it. Those three pieces are rewritten against version 3, everything else in both is untouched, and the learners who had proven the 14-day rule get a five-minute refresher.

Two honest limits. The system does not know whether a document is up to date; it records processing dates and content hashes, flags citations whose source file was deleted, and keeping documents current stays the customer's job. And training content built with no documents draws on the model's general knowledge plus optional web search; the teaching checks still run, but the footnote view shows that content as unsourced, and we say that plainly to buyers.

We do not measure completion; we measure objectives. Every quiz question exists because an objective requires it, and every event that says something about ability lands in an append-only evidence ledger: quiz answers at full weight, AI-graded practice at reduced weight because the grader is a model, interactive elements that require a correct answer, and questions asked of the tutor at weight zero, so asking for help never lowers a score but keeps the learner's own words for spotting misconceptions. Engagement proves attendance, never ability.

Mastery per skill is a running Bayesian estimate over that ledger. The math:

The formula set is versioned and the ledger is the source of truth, so a better formula can replay every learner's history with no migration and no lost data. The closest published cousins are Bayesian Knowledge Tracing and Elo-style calibration; ours adds continuous evidence weights, explicit forgetting, a separate uncertainty axis, and full replayability. After three wrong answers on one skill, the learner's wrong choices are analysed, the recurring misconception is named in plain words, matched against the workspace's known misconceptions so the same confusion is not stored under ten labels, and closed automatically once mastery recovers.

Acting on the estimate

Every recommendation is deterministic arithmetic over the belief, so it is explainable and repeatable. A detector compares three maps kept in one vocabulary: what roles and documents require, what the training content teaches (approved links only), and what the people can demonstrably do. Mismatches become findings of six kinds: build (required, not taught), retire (taught, no longer required), assess (taught, nothing measures it), reinforce (taught and measured, the group is still below the bar), refresh (a group whose mastery decayed under the bar from time alone), and misconception (three or more learners share a named wrong idea the content does not address). No model decides what to teach; the detector is arithmetic. Findings are ranked by demand, severity and freshness, with a cap per skill so one skill cannot flood the queue. When a weak skill rests on a weaker prerequisite, the recommendation targets the foundation, walked over a cycle-guarded prerequisite graph. A low-confidence gap gets an assessment before training content, because measuring beats teaching when the estimate itself is the problem.

Every finding waits for a human. An accepted finding writes a brief by rule (which skills, which documents demanded them, which misconception to fix, what the audience already masters and must not be re-taught) and that brief runs through the full pipeline above, so the new training content is linked to its skills from the start. Formally:

Feedback from admins and learners enters at five depths, and all of it is stored outside the writer models:

• Per asset: thumbs up or down. Admins already see a confidence and relevance score on every asset in edit mode. The next control is a verdict on each individual asset, not the page, not the finished training content. Each verdict is stored in a judgement ledger next to what the writer was given and what it produced, which makes every thumbs down a labelled training example.
• Per build. Reviewer findings redo the failing component; edits at the structure gate rewrite every page against the edited structure.
• Per workspace, permanently. An approved definition rejects contradicting content from then on. A misconception named from real learner answers becomes the brief for the training content that fixes it. Accepted and dismissed findings reweight what the system proposes next: the acceptance rate per kind of finding becomes a ranking weight, pulled toward neutral when few decisions back it, and a dismissed finding sleeps for six months unless it clearly worsens. The system stops proposing what the organisation keeps refusing.
• Per product, permanently. Recurring client feedback becomes a code check that runs on every piece of training content for every customer. "A practice section must contain something to do" started as a complaint and is now a check. We ship improvements on this channel weekly.
• Into our own models. The writers stay on rented frontier models, on purpose: they are replaced every few months, and everything we teach them would be lost. What becomes ours are the judges, and there is a concrete pipeline for that, described below.

The judge pipeline

A decision log records every skill-linking decision: the content as judged, the candidate skills shown, the model's answer, and the admin's later yes or no on each candidate. A dataset builder turns the human-decided rows into training examples, holds the newest fifth aside for testing, and refuses to train until a few thousand human-decided examples exist across several workspaces. A training run then fine-tunes a small open-weights model with LoRA on rented GPUs, saves the weights to persistent storage, and scores itself against the vendor model on the same held-out decisions: how often it is right, and whether its confidence is honest. A serving endpoint runs the model, scales to zero when idle, and speaks the same protocol as the vendors, joining the fallback chain as our judge first, the vendor when ours is down. Shadow first: the judge answers alongside the vendor without acting until it agrees with humans at least as often as the vendor does, then takes over one workspace at a time. Citation checking and the glossary check are the next two judges on the same log, training and serving pipeline. The per-asset thumbs verdicts feed the same ledger.

The practical result: the writer models are interchangeable, and everything we learn from customers is stored where future builds read it, so quality improves with use instead of depending on which model is behind the API that day.

Model choice is a setting. Every job names its model, routed across three vendors with health checks and per-call failover; stronger models where judgment matters, fast cheap models everywhere else. We leave exact model names out on purpose, because they change every few months. Everything accumulated sits outside the model: the knowledge base, learning modes, plan rules, schemas, review rules, human corrections and the judgement ledger.

All of it applies to whichever model answers the API today. The schema and review checks also act as the drift detector: a model that starts behaving differently fails them and is retried or repaired. Because the judges train from our own ledger, a new base model on our side means one retraining run. One honest gap: there is no automatic test suite yet that builds a fixed set of training content on each model change and compares them; today a swap is checked by building training content and reading it, and a set of fixed test briefs is the next step.

The system is built so that generation is cheap and replaceable, while the parts that hold the value are ours: the pedagogy configuration, the verification checks, the knowledge extracted from each customer's documents, the evidence ledger about each learner, and the record of every human decision. Copying our output does not copy any of that, and each of these took months to design and debug against real customers. Every piece of training content built and every decision made adds to it.