omnibase/docs

The back office's manual

With omnibase/admin installed, /docs shows the documentation inside the back office: the page tree and the page's headings in a rail beside the text, and a search palette (Ctrl K) that matches in the browser.

# config/routes.yaml
wikidoc_controller:
    resource: "@WikidocBundle/src/Controller"
    type: attribute
    prefix: /

# config/packages/wikidoc.yaml
wikidoc:
    # Documentation roots, LOWEST priority first.
    roots:
        base:
            path: '%base.package_dir%/docs'
            label: 'Base'
        app:
            path: '%kernel.project_dir%/docs'
            label: 'Application'

Roots override each other by path

A page is identified by its path inside a root. When two roots hold install/requirements.md, the later one wins and the manual shows a badge on it: the rule Symfony uses for a bundle's templates. A root that does not exist on disk is skipped - shipping the configuration before writing any page is harmless.

The hierarchy is the folder tree. A folder is a section; its index.md, if it has one, is the section's own page. The root's index.md is the manual's home. Titles and order: see Writing pages.

Routes

Raw HTML in a page is escaped: these files are rendered on an authenticated page, and markup that Markdown cannot write is refused rather than run.

In code

use Base\Wikidoc\Documentation\DocumentationRegistry;

$registry->getTree();                 // DocPage[]: the top-level pages and sections
$registry->get('install/requirements');
$registry->getDefault();              // the home page, else the first page
$registry->getSequence();             // every page in reading order
$registry->getNeighbours($page);      // [previous, next]
$registry->getAncestors($page);       // its breadcrumb

Without Symfony

The documentation itself loads without the framework: the bundle's root namespace - Base\Wikidoc\Documentation (the roots, their pages, the Markdown), Base\Wikidoc\Manual (the public manuals, their versions, the export of a branch) and Base\Wikidoc\Search (the search, its index, the Typesense client's interface) - needs PHP, league/commonmark, Symfony's standalone Finder and Filesystem (and Process for an export), a PSR-6 pool if one is given, nothing else. tests/BareTest.php loads its classes (tests/Fondation.php) with Composer's autoloader alone and fails if a class or a file of the framework comes with them.

The services by hand

use Base\Wikidoc\Documentation\DocumentationRegistry;
use Base\Wikidoc\Documentation\MarkdownRenderer;
use Base\Wikidoc\Manual\ManualMarkdownRenderer;
use Base\Wikidoc\Manual\ManualRegistry;
use Base\Wikidoc\Manual\PageRenderer;
use Base\Wikidoc\Search\LocalSearch;
use Base\Wikidoc\Search\ManualIndex;
use Base\Wikidoc\Search\ManualSearch;

// The back office's pages of a root, and one of them in HTML.
$pages = new DocumentationRegistry(['app' => ['path' => __DIR__.'/docs', 'label' => 'Application']]);
$html = (new MarkdownRenderer())->renderFile($pages->getDefault()->file);

// The public manuals found under a folder of repositories, a page, the search.
$manuals = new ManualRegistry([], ['/srv/repos/*/*'], ['export_dir' => '/srv/exports']);
$manual = $manuals->get('glitchr/omnitrade');
$version = $manual->getVersion(null);
$page = (new PageRenderer($manuals, new ManualMarkdownRenderer()))->render(
    $manual, $version, $manuals->pages($manual, $version)->get('installation'),
    fn (string $target, ?string $anchor = null) => '/docs/'.$manual->urlPath($version->name, $target).($anchor ? '#'.$anchor : ''),
    fn (string $file) => '/docs/_file/'.$manual->urlPath($version->name, $file),
);
$hits = (new ManualSearch($manuals, new LocalSearch(new ManualIndex($manuals, new MarkdownRenderer()))))->search('webhook');

ManualRegistry's three arguments are what wikidoc.manuals, wikidoc.discover and the other options give it under Symfony (Configuration reference): the manuals named, the glob patterns to discover, and export_dir, branches, docs, readme, remote, cache_ttl, exclude. PageRenderer::render() takes two closures, the address of a page (with its anchor) and the address of a file of the repository: the host's routes.

Run in the core's harness

tests/bare/docs-without-framework.php does the same over the tests' fixtures (tests/Fixtures/repos/*/*, three repositories). Run with the harness's autoloader and this checkout's classes (tests/bare/autoload.php), it returns:

array (
  'manuals' =>
  array (
    0 => 'acme/plain',
    1 => '@acme/script',
    2 => 'acme/widget',
  ),
  'link' => true,
  'picture' => true,
  'engine' => 'local',
  'hit' =>
  array (
    0 => '20-guides/webhooks',
    1 => 'Webhooks',
    2 => 'Signature',
    3 => 'signature',
  ),
  'pages' =>
  array (
    0 => 'installation',
    1 => '20-guides/events',
    2 => '20-guides/webhooks',
  ),
)

The three manuals found (an npm package keeps its scope), a relative link and a picture of a page rewritten to the closures' addresses, a search answered by the local index with its section and anchor, the back office's pages of a root in reading order.

Under Symfony

What needs Symfony is under src/Bridge/Symfony: the bundle, its configuration and extension, the two controllers, the two commands and the Typesense client of glitchr/typesense-bundle (Reference). The former names still answer: Base\Wikidoc\WikidocBundle in config/bundles.php, the folders @WikidocBundle/src/Controller and @WikidocBundle/src/Site/Controller in the routes, Base\Wikidoc\DependencyInjection\*, Base\Wikidoc\Command\*.

Public manuals

One manual per package, read where the package is written: the docs/ folder and the README of its repository. One version per branch. The pages are rendered on the public site, the way symfony.com/doc renders its own.

Switching them on

# config/packages/wikidoc.yaml
wikidoc:
    public:
        enabled: true
        path: docs            # the address they are published under
    manuals:
        glitchr/omnitrade:
            path: '%env(OMNI_REPOS)%/omnitrade/core'    # the repository's checkout
    # or: every package found under these folders
    discover:
        - '%env(OMNI_REPOS)%/*/*'

# config/routes.yaml
wikidoc_manuals:
    resource: "@WikidocBundle/src/Site/Controller"
    type: attribute
    prefix: /

The public controller is in src/Site/Controller, outside src/Controller: a site that imports the latter for the back office's manual does not get the public routes with it.

What a manual is made of

A package without a docs/ folder and with a README is a manual of one page; a package with neither is not a manual.

Naming them, or finding them

wikidoc.manuals names manuals one by one, keyed by the package's name:

wikidoc.discover takes glob patterns: every folder matched that holds a composer.json or a package.json and something to read is a manual named after its package. wikidoc.discover_exclude leaves packages out, by name or by folder (fnmatch patterns). A manual named in wikidoc.manuals wins over one discovered under the same name. A folder that is not there - not mounted, not cloned yet - is skipped.

The folders are looked at when a page is asked for, not when the container is compiled: a repository cloned later is simply there. wikidoc.cache_ttl (seconds, 0 by default) keeps the list of discovered folders, and the local search records, for a while.

Versions

A version is a branch of the repository whose name matches wikidoc.branches (by default 1.x, 2.0, main, master...).

• The branch checked out is read where it is: edit a file, reload the page. It is the manual's default version.
• The other branches are copied by a command, with git (ls-tree, show: the repository is only read, a read-only mount is enough):

bin/console wikidoc:sync                    # every manual
bin/console wikidoc:sync glitchr/omnibus    # one
bin/console wikidoc:sync --remote=github    # read another remote's branches beside the local ones

It writes each branch's docs/ and README (Markdown and pictures) under wikidoc.export_dir (var/wikidoc/<package>/<branch>/), where the next request finds it as a version, and removes the copy of a branch that is gone. Local branches come first, then the remote's (origin): a branch that was fetched but never checked out is a version too. Without git or symfony/process the command says so, and each manual keeps the one version of its working tree.

Run it after a deployment or a fetch, or from cron.

Versions with a checkout of their own, when a site prefers worktrees to copies:

wikidoc:
    manuals:
        glitchr/omnibus:
            path: '/srv/repos/omnibus/core'
            versions:
                '2.x': '/srv/repos/omnibus/core'
                '1.x': { path: '/srv/worktrees/omnibus-1.x', branch: '1.x' }

Versions are listed newest first (main and master ahead, then 3.x, 2.x, 1.x). A folder that is not a git repository has one version, named current.

Addresses

A manual's part of an address is its package name, lower case, without the @ of an npm scope: glitchr/stickyjs for @glitchr/stickyjs. One address per page: an address without a version redirects to the one that spells it out. A folder without an index.md redirects to its first page.

<a href="{{ path('wikidoc_manual', {path: manual.urlPath()}) }}">{{ manual.label }}</a>
<a href="{{ path('wikidoc_manual', {path: manual.urlPath('1.x', 'webhooks')}) }}">Webhooks, 1.x</a>

What a page shows

• the manual's contents, the current branch open;
• a breadcrumb: Documentation / the manual / the version / the sections / the page;
• the version menu, each entry going to the same page in that version (to its home when that version has no such page), and a notice on a page of a version that is not the default one;
• the page, its headings carrying anchors;
• "On this page": the page's second and third level headings, the one being read marked;
• "Edit this page", to the file on its forge, and the path of the source file;
• previous and next, in reading order.

"Edit this page" is {repository}/edit/{branch}/{file} (GitLab: {repository}/-/edit/...); wikidoc.edit_url sets another pattern with the same three placeholders. A manual whose repository is not known has no such link. The source stays the Markdown in the repository: there is no editor here.

In code

use Base\Wikidoc\Manual\ManualRegistry;

$manuals->all();                               // array<string, Manual>, keyed "glitchr/omnitrade"
$manual = $manuals->get('glitchr/omnitrade');  // or null
$manual->versions;                             // array<string, ManualVersion>, newest first
$manual->getDefaultVersion();
$manual->urlPath('1.x', 'webhooks');           // "glitchr/omnitrade/1.x/webhooks"
$manuals->pages($manual, $version);            // a DocumentationRegistry: tree, pages, reading order
$manuals->resolve('glitchr/omnitrade/1.x/webhooks');   // ManualLocation: manual, version, path

ManualRegistry and ManualSearch are registered whether the public controller is or not: a site may read manuals from controllers of its own.

Writing pages

A page is a Markdown file (CommonMark with GitHub's extensions: tables, strikethrough, task lists, autolinks). It should read well on its forge too: nothing here needs a syntax GitHub does not render.

Title and order

---
title: Installation
order: 10
---

# Installing the widget

The title in the contents is the front matter's title, else the first # heading, else the file's name. The order is the front matter's order, else a number the file's name starts with (10-getting-started.md, 20-architecture/), else 500; pages of the same order are sorted by title. The front matter is not printed.

A page without a first-level heading gets its title printed above it.

Links

Write links for the repository; the manual follows them:

A link is relative to the file it is written in, wherever that file is: a README links to docs/installation.md. A link that climbs out of the repository is left as written. Pictures served: png, jpg, gif, svg, webp, avif.

Callouts

Written as GitHub writes them - a quote whose first line is a marker:

> [!NOTE]
> The widget needs PHP 8.2.

> [!WARNING]
> Never commit the signing secret.

[!NOTE], [!TIP], [!IMPORTANT], [!WARNING], [!CAUTION]. Their titles are translated (wikidoc.callout.*: "Astuce", "Attention"...). The back office's manual renders them as plain quotes.

Code

```php
$widget = new Widget(secret: 'not-a-real-secret');
```

A fenced block is framed, named after its language, and has a "copy" button. With tempest/highlight installed it is coloured on the server - php, yaml, twig, sh / bash, js, json, html, css, scss, sql, xml, ini, dotenv, nginx, dockerfile, diff, and what else the library knows; a language it does not know is shown plain. Without the library every block is plain.

Raw HTML is escaped, in every page.

Headings

Second to fourth level headings get an anchor (## Signature is #signature), linked from "On this page" (second and third level) and from the search results.

Search

Every page of the public manuals has a search field; Ctrl K (or /) opens it as a palette that asks the server as one types, in the manual being read first, in every manual on demand. Without JavaScript the field is a form that goes to the results page.

/docs/_search?q=webhook                                   every manual, each in its default version
/docs/_search?q=webhook&manual=glitchr/omnitrade          one manual, its default version
/docs/_search?q=webhook&manual=glitchr/omnibus&version=1.x
/docs/_search?q=webhook&format=json                       what the palette reads

A result is a page, or a section of a page (it links to the heading). The answer says who gave it: engine is typesense or local.

The local index

With nothing configured, the search reads the pages' own index: one record per page and one per heading - title, section, text (code included: a class name or a command is what people search for) - built by SearchIndexBuilder, the index of the back office's palette, and matched in PHP. Every word of the query must be found; a title outranks a heading, a heading the text. No typo tolerance.

Typesense

composer require glitchr/typesense-bundle

# config/packages/typesense.yaml: the connection (no entity mapping is needed)
typesense:
    default_connection: default
    connections:
        default:
            secret: '%env(TYPESENSE_SECRET)%'
            scheme: '%env(TYPESENSE_SCHEME)%'
            host: '%env(resolve:TYPESENSE_HOST)%'
            port: '%env(int:TYPESENSE_PORT)%'
            options:
                connection_timeout_seconds: 1

# config/packages/wikidoc.yaml
wikidoc:
    search:
        typesense:
            enabled: true
            connection: ~        # typesense-bundle's connection; null: its default one
            prefix: wikidoc      # the collections are <prefix>_<manual>_<version>

bin/console wikidoc:index                      # every manual, every version
bin/console wikidoc:index glitchr/omnitrade    # one

One collection per manual and version (wikidoc_glitchr-omnitrade_1-x), dropped and written again at each run, with the same records as the local index:

Run without a manual's name, the command also drops the collections of manuals and versions that are no longer published. Run it after wikidoc:sync, after a deployment, or from cron.

A search over every manual is one multi_search call, one search per manual (40 per call).

When the engine does not answer

The local index answers, and the result says local:

• glitchr/typesense-bundle is not installed, or enabled is false;
• the server is down or times out (keep connection_timeout_seconds short);
• a manual that is searched has no collection yet (wikidoc:index has not run since it appeared).

Nothing is shown to the visitor but the results; a notice goes to the log. wikidoc:index itself exits with an error when the engine is switched on and does not answer, and says so when it is switched off.

In code

use Base\Wikidoc\Search\ManualSearch;

$result = $search->search('webhook');                                // every manual
$result = $search->search('webhook', 'glitchr/omnitrade', '1.x', limit: 10);

$result->engine;     // "typesense" or "local"
$result->found;
foreach ($result->hits as $hit) {
    $hit->manual; $hit->version; $hit->path;      // "glitchr/omnitrade", "1.x", "webhooks"
    $hit->title; $hit->section; $hit->anchor;     // "Webhooks", "Signature", "signature"
    $hit->snippet;
}

Base\Wikidoc\Search\TypesenseClientInterface is the handful of calls made to the engine; a test replaces it with its own (see tests/Search/FakeTypesense.php).

Templates and look

The public pages extend the host's layout1.html.twig and fill its title, description, stylesheets and content blocks, as the other omnibase bundles do.

A site overrides one by placing a file of the same name under templates/bundles/WikidocBundle/client/ - typically index.html.twig, to list the manuals the way its own catalogue groups them.

The look is the site's

public/css/manual.css (bundles/wikidoc/css/manual.css once assets:install ran) has no colour and no typeface of its own. A site sets custom properties, in its light and its dark mode:

:root {
    --manual-ink: #101418;        /* text */
    --manual-soft: #566070;       /* secondary text */
    --manual-line: #d5dae1;       /* rules, borders */
    --manual-bg: #ffffff;         /* the page */
    --manual-surface: #f2f4f5;    /* code, hovered rows */
    --manual-accent: #1238d6;     /* links, the current page */
    --manual-font: "Recursive", sans-serif;
    --manual-font-display: "Archivo", sans-serif;
    --manual-font-mono: "Recursive", monospace;
    --manual-top: 5rem;           /* the height of a sticky header: where the side columns stick */
}

Also: --manual-on-accent, --manual-mark (the searched words), --manual-radius, --manual-code-bg, --manual-code-ink, the code's colours --manual-hl-keyword, -type, -property, -value, -number, -comment, -variable, and the callouts' --manual-note, -tip, -warning, -caution.

The script

public/js/manual.js has no dependency: the search palette, the "copy" buttons, the current heading in "On this page", the contents folded behind a button on a narrow screen. The pages load it themselves; a site that bundles its scripts may import it instead - loading it twice does nothing - and it sets itself up again after a page is swapped in by transparentjs (transparent:load).

Without the script everything is still reachable: the search field posts to the results page, the version menu is a <details>, the contents are in the page.

Texts

translations/wikidoc+intl-icu.{en,fr}.yaml: @wikidoc.manual.*, @wikidoc.search.*, @wikidoc.callout.*.

Configuration reference

config/packages/wikidoc.yaml, as Base\Wikidoc\Bridge\Symfony\DependencyInjection\WikidocConfiguration declares it (bin/console config:dump-reference wikidoc in the core's harness, this checkout mounted over vendor/omnibase/docs):

wikidoc:

    # Documentation roots, lowest priority first; a later root overrides same-path pages.
    roots:

        # Prototype
        name:
            path:                 ~ # Required
            label:                null

    # The manuals: one per package, read in the docs/ folder and the README of its repository. Keyed by the package's name.
    manuals:

        # Prototype
        name:

            # The repository's checkout.
            path:                 ~ # Required
            label:                null
            description:          null

            # The Markdown folder inside the repository (default: wikidoc.docs).
            docs:                 null

            # The README inside the repository; false: none (default: wikidoc.readme).
            readme:               null

            # Its web address, for "Edit this page". Default: the checkout's origin remote.
            repository:           null
            default_version:      null

            # Versions with a checkout of their own: {"2.x": {path: ..., branch: "2.x"}}. Default: the branch checked out, and those wikidoc:sync exported.
            versions:

                # Prototype
                name:
                    path:                 ~ # Required
                    branch:               null

    # Glob patterns: every folder matched that holds a composer.json or a package.json, and a docs/ folder or a README, is a manual named after its package.
    discover:             []

    # Package names (or folders) left out of the discovery; fnmatch patterns.
    discover_exclude:     []

    # The Markdown folder inside a repository.
    docs:                 docs

    # The README inside a repository; false: none.
    readme:               README.md

    # The git remote a repository is published on.
    remote:               origin

    # The branches that are versions.
    branches:             /^(\d+\.(x|\d+)|main|master)$/

    # Where wikidoc:sync copies the branches that are not checked out.
    export_dir:           '%kernel.project_dir%/var/wikidoc'

    # "{repository}/edit/{branch}/{file}"; null: the forge's own convention.
    edit_url:             null

    # Seconds the discovered folders and the local search records are kept; 0: read again at each request.
    cache_ttl:            0
    public:

        # Registers the public controller (import @WikidocBundle/src/Site/Controller in the routes).
        enabled:              false

        # The address the manuals are published under.
        path:                 docs
    search:
        typesense:

            # Needs glitchr/typesense-bundle; without it, or with the server down, the local index answers.
            enabled:              false

            # The typesense-bundle connection; null: its default one.
            connection:           null

            # The collections are <prefix>_<manual>_<version>.
            prefix:               wikidoc

The roots are given to DocumentationRegistry and the manuals' options to ManualRegistry as arguments (their paths resolved: %kernel.project_dir%, an %env()%); the other options are also parameters (wikidoc.public.path, which the public routes read, wikidoc.cache_ttl...).

Reference

Routes

<public.path> is wikidoc.public.path (docs). The back office's controller is registered only when omnibase/admin is installed (it renders in its layout, with its AdminContext and MenuBuilder); the public one only with wikidoc.public.enabled. Both share /docs by default: a site that publishes its manuals and has the back office's manual gives public.path another value, or imports the two under different prefixes.

Console commands

Description:
  Copies the documentation of every version branch of the manuals' repositories

Usage:
  wikidoc:sync [options] [--] [<manual>]

Arguments:
  manual                One manual, by its package name (default: all of them)

Options:
      --remote=REMOTE   The remote whose branches are read beside the local ones [default: "origin"]

Description:
  Writes the manuals' search index into Typesense

Usage:
  wikidoc:index [<manual>]

Arguments:
  manual                One manual, by its package name (default: all of them)

What each does is in Public manuals (wikidoc:sync) and Search (wikidoc:index). Run in the core's harness, where no manual is configured and Typesense is off, they answer:

$ php bin/console wikidoc:sync

 ! [NOTE] No manual is configured (wikidoc.manuals, wikidoc.discover).

$ php bin/console wikidoc:index

 ! [NOTE] Typesense is not switched on (wikidoc.search.typesense.enabled,
 !        glitchr/typesense-bundle): the search reads the local index - 0
 !        record(s) in 0 manual(s).

Services

config/services.php registers DocumentationRegistry (its roots from wikidoc.roots), MarkdownRenderer, SearchIndexBuilder, ManualRegistry, ManualMarkdownRenderer, PageRenderer (its $editPattern from wikidoc.edit_url), ManualIndex, LocalSearch, ManualSearch, BranchExporter, the two commands and the controllers. With search.typesense.enabled and glitchr/typesense-bundle installed, the extension adds Bridge\Symfony\Search\BundleTypesenseClient (on typesense_manager and the connection named), aliased as Base\Wikidoc\Search\TypesenseClientInterface, and Search\TypesenseIndex; without them ManualSearch has no engine and the local index answers.

Testing

# in omnisense/omnibase's harness (core/docker)
docker compose -f compose.yml -f compose.checkouts.yml run --rm omnibase test docs
# or from an application that has the bundle
php vendor/bin/phpunit -c vendor/omnibase/docs/phpunit.xml.dist

The suite works on fixtures: tests/Fixtures/repos/acme/{plain,script,widget}, three repositories (a README only, an npm package, a manual with guides, a picture and versions), and tests/Fixtures/export, the copies wikidoc:sync would make.

An application's own test of its manual can build DocumentationRegistry on its docs/ and assert that every page has a title and that getSequence() is in the order it wants; a search of its own goes through ManualSearch with LocalSearch, or a TypesenseClientInterface of its own in place of a server.