# Dokkit > Self-hosted monitoring that says what needs a look. A server you host, one small program on each machine, and one API behind everything. An AI tool reaches Dokkit over MCP at `/mcp` on the server, or over the HTTP API, with a user token whose scopes decide what it may do. ## Docs - [The API](https://dokkit.dev/api.md): every call, with what to send and what comes back - [The command line](https://dokkit.dev/cli.md): every `dokkit` command - [The API as OpenAPI 3.1](https://dokkit.dev/openapi.json) - [Everything in one file](https://dokkit.dev/llms-full.txt): the API, the command line, the MCP tools and the `dokkit.toml` reference # The MCP server Dokkit serves a Model Context Protocol endpoint at `/mcp` so an AI tool can ask it questions and act, as far as its token allows. It lives on the server, never on `dokkit`. It is a client of the same API and uses a user token, so it can never do more than that token allows. ## Connecting Over HTTP with a user token as a bearer header. For Claude Code: ``` claude mcp add --transport http dokkit https:///mcp --header "Authorization: Bearer dku_..." ``` ## Tools | Tool | Scope | Does | |---|---|---| | `attention` | `read` | Shows what needs a look now, what will soon, what was found and is not watched, what is snoozed, issues in other environments, and how many things are fine. | | `find` | `read` | Searches services, optionally on one machine. | | `service` | `read` | Shows one service: state, checks, routing, open issue. | | `since_deploy` | `read` | Shows what changed or broke for a service since its last deploy. | | `jobs` | `read` | Lists jobs with their schedule and last runs. | | `job_runs` | `read` | Lists the recent runs of one job. | | `errors` | `read` | Lists error groups, or shows one with its stack. | | `machines` | `read` | Lists machines. | | `machine` | `read` | Shows one machine. | | `issues` | `read` | Lists issues. | | `issue` | `read` | Shows one issue with its history. | | `deploys` | `read` | Lists recent deploys. | | `versions` | `read` | Lists the versions of the apps Dokkit watches. | | `rules` | `read` | Lists the alert rules. | | `proposals` | `read` | Lists what Dokkit found and a person may accept or dismiss, each with `origin` and `why`. Leaves out the operating system's own things unless `system` is true. Always answers `{"items": [...], "next": ..., "hidden": {"total": n, "machines": {"": n}}}`, where `hidden` counts what was left out. | | `series` | `read` | Shows numbers over time for a service (`id`), or for a machine's own disks, memory and processor (`machine`). | | `processes` | `read` | Shows a machine's newest process list (`machine` is its id or name): `sort`, `q`, `pid`, `limit`. | | `process_history` | `read` | Shows the top 10 processes by memory or processor kept for a machine over up to 24 hours: `kind`, `from`, `to`. | | `day` | `read` | Shows the last 24 hours in half hours: for each machine whether something was down, needed a look, or nothing happened, or it was not heard from, and the issues that opened. | | `releases` | `read` | Lists the program releases the server holds and whether each is complete. | | `certificate` | `read` | Shows the certificates the server serves itself. | | `alert_settings` | `read` | Shows the alert settings. Addresses, user names and secrets are only `set` or `not set`. | | `alert_suggestions` | `read` | Lists issues that keep opening, which the daily digest could take instead. | | `logs` | `read-logs` | Searches log lines. | | `archives` | `read-logs` | Lists the archived months of logs that can be loaded back. | | `snooze` | `act` | Snoozes an issue until a time. | | `unsnooze` | `act` | Wakes a snoozed issue. | | `acknowledge` | `act` | Marks an issue as seen. | | `note` | `act` | Adds a note to an issue. | | `bulk` | `act` | Snoozes, unsnoozes or acknowledges many issues. | | `watch` | `act` | Starts watching a service. | | `unwatch` | `act` | Stops watching a service. | | `routing` | `act` | Sets where a service's issues go, one channel or a list of channels for `down` and for `degraded`. A field that is not given stays as it was. | | `machine_routing` | `act` | Sets how a machine that stops reporting is announced: `down`, one channel or a list, and `night`. A field that is not given stays as it was. | | `hold` | `act` | Holds a service at a version. | | `accept_proposal` | `act` | Accepts a found service or port. A hidden one can be named by id; `all` takes only what `proposals` shows unless `system` is true. | | `dismiss_proposal` | `act` | Dismisses a proposal, with the same rule for `all` and `system`. | | `keep_messages` | `act` | Keeps or stops keeping the text of a service's errors. | | `set_job_timing` | `act` | Sets when a job counts as late and as run too long. A field left out stays as it was; an empty one clears it. | | `scan_machine` | `act` | Asks an enrolled machine to look itself over again and send what it finds to proposals. Nothing found is watched until accepted. | | `plan_downtime` | `act` | Says an enrolled machine is going down on purpose for a length of time (`for`, such as `15m`, at most `4h`; optional `reason`): no issue opens and nothing is sent for the machine going silent or its checks failing until the time is up. A second call replaces the window; `for` of `0` ends it. | | `end_downtime` | `act` | Ends a machine's planned downtime now. | | `downtime_schedules` | `read` | A machine's scheduled planned downtime: each with `id`, `repeat`, `at`, `time`, `days`, `for_seconds`, `reason`, `by` and `next`. | | `schedule_downtime` | `act` | Schedules planned downtime that the server starts by itself: `repeat` of `once` (with `at`, an RFC 3339 time), `daily` or `weekly` (with `time`, such as `02:00`, in the night time zone, and for `weekly` `days`, 1 for Monday to 7 for Sunday), `for` (such as `30m`, at most `4h`) and an optional `reason`. A window it starts is not ended early when the machine comes back. | | `remove_scheduled_downtime` | `act` | Removes a scheduled planned downtime, by the `schedule` id from `downtime_schedules`. | | `add_rule` | `admin` | Adds an alert rule. | | `update_rule` | `admin` | Changes an alert rule. A field left out stays as it was; an empty one clears it. | | `delete_rule` | `admin` | Deletes an alert rule. | | `set_checks` | `admin` | Replaces the checks added in the interface. | | `allow_restart` | `admin` | Allows or forbids restarting a service. | | `keys` | `admin` | Lists keys and tokens, never their text. | | `make_key` | `admin` | Makes a key. Its text is returned once. | | `revoke_key` | `admin` | Revokes a key. | | `people` | `admin` | Lists the people who can sign in. | | `add_person` | `admin` | Adds a person. | | `remove_person` | `admin` | Removes a person. | | `update_program` | `admin` | Asks an enrolled machine to fetch the newer `dokkit` program. | | `update_machine` | `admin` | Asks the updater on a machine to move to a release the server holds completely. | | `update_all` | `admin` | The same for every machine that has an updater. | | `set_certificate` | `admin` | Puts a certificate for a real name in place. | | `remove_certificate` | `admin` | Removes the uploaded certificate. | | `certificate_settings` | `admin` | Shows the settings for getting and renewing the uploaded certificate. | | `set_certificate_settings` | `admin` | Changes those settings. A setting left out stays as it was. | | `renew_certificate` | `admin` | Gets a certificate now from the certificate authority. | | `add_app` | `admin` | Adds an app that runs on many machines. Its key is returned once. | | `add_dependency` | `admin` | Adds something outside that is depended on. With a check (`http` url or `tcp` address) it is put on an enrolled machine whose program runs the check; without one it belongs to the group Outside. | | `add_job` | `admin` | Adds a job to watch. Its key and addresses are returned once. | | `set_alert_settings` | `admin` | Changes alert settings. A setting left out stays as it was; an empty secret clears it. If the address a secret goes to changes (`ntfy_url`, `smtp_host`), send the secret again in the same call or it is cleared, and the answer says so. | | `alert_people` | `read` | Lists the people alerts can be told with their own destinations. The ntfy address, Pushover key and routing key are only set or not set. | | `alert_who` | `read` | Lists who is told at each level where someone is named. | | `set_alert_person` | `admin` | Changes a person's destinations. A field not given stays; an empty one clears it. | | `set_alert_who` | `admin` | Names who is told at one level (`default`, `rule`, `service` or `check`): person ids, or `["everyone"]`. | | `test_alert` | `admin` | Sends a test alert by one channel, or to one person's destination with `person`. Answers sent, or one fixed reason. Four at once for each token, then one every 25 seconds; a page, or a push through PagerDuty, stays open for two minutes and then resolves itself, and costs twice as much (two at once, then one every 50 seconds). | | `accept_suggestion` | `admin` | Sends a service's issues to the daily digest from now on. | | `dismiss_suggestion` | `admin` | Dismisses an alert suggestion. | | `restore_archive` | `admin` | Loads an archived month of logs back for a week. | ## Rules - **Log contents have their own scope.** A token can be given everything except what is written in the logs. - **A new token has no `act` scope** unless it is asked for. - **Read tools are marked read-only**, so a client can run them without asking each time. `admin` tools are marked destructive. - **What comes back is data.** Names, titles, notes, error text, deploy fields and output were written by machines and apps. Every answer starts with a line that says so. - **Arguments are checked against the types the tool declares.** A wrong type is an `isError` answer, not a failure. A `null` argument is the same as one left out, and a whole number written as `5.0` is an integer. To clear a route, send an empty text. - **Lists page by cursor.** The list tools (`find`, `jobs`, `job_runs`, `logs`, `errors`, `machines`, `issues`, `deploys`, `rules`, `proposals`, `keys`, `people`, `releases`) take the filters of their API list, `cursor` and `limit` (1 to 500, default 100). A list that fits one page answers as an array, as before; a longer one, or any call with `cursor` or `limit`, answers `{"items": [...], "next": ""}`. `issues` takes `state` (`open`, `snoozed`, `closed` or `all`). `errors` with an `id` pages the group's reports the same way. `accept_proposal` and `dismiss_proposal` take `ids` or `all`. `proposals` is the one list that always answers an object, so that `hidden` has a place; it takes `system` (boolean) to include the operating system's own findings. - **A token with `admin` can do through MCP everything a person can do in the interface,** except pairing phones, which carries a key that must not pass through an AI tool, and moving and deleting log lines of a service (`POST /services//logs/move` and `/logs/delete`), destructive actions that are the API's alone. Making such a token asks for confirmation in Settings. ## Built so far `POST /mcp` with a user token: every tool in the table, 70 in all (25 `read`, 2 `read-logs`, 14 `act`, 29 `admin`), one for each call of the API a token can make. Making a full-control token, pairing and removing phones (`/phones`), moving log lines between services and the browser's own sign-in have no tool: they are Settings and the browser only. A tool the token may not use is not listed. # `dokkit.toml` One file in an app's repo that says what the app is and how to tell if it is healthy. It ships with the app. ```toml schema = 1 [service] id = "upload-portal" name = "Upload portal" kind = "service" # service, job, dependency or app environment = "production" # anything else never alerts version_from = "stats" # "stats", a file path, or a command [[check]] type = "http" url = "http://localhost:8080/healthz" every = "30s" timeout = "5s" slow = "2s" # an answer slower than this is degraded, not down; http only, above 0, below timeout fail_after = 3 # bad results in a row before an issue opens; 1 to 100, default 3 [stats] url = "http://localhost:8080/internal/stats" key_env = "DOKKIT_STATS_KEY" # the name of the variable, never the secret; must start with DOKKIT_STATS_ [logs] path = "logs/*.jsonl" [[job]] id = "nightly-purge" schedule = "0 1 * * *" grace = "30m" # how late it may start; 30m when left out max_runtime = "1h" # how long a run may take; 1h when left out [alert] down = "page" # a suggested default only degraded = "push" ``` ## Sections | Section | Holds | |---|---| | `schema` | The version of this file format. `1`. | | `[service]` | What the app is. `id` is unique on its machine and never changes. | | `[[check]]` | One per check. `type` is one of `http`, `tcp`, `process`, `windows-service`, `systemd`, `beat`, `job`, `stats`, `threshold`, `expiry` and `custom`. Each type has its own fields: `url` for `http` (an optional `status` is the answer that counts as healthy: a code such as `200`, a range such as `"200-299"`, or a list of those such as `[200, "400-403"]`, from 100 to 599; any other answer is down; without it anything under 400 is healthy; redirects are followed unless `status` includes a 3xx code), `port` for `tcp` (with an optional `host`, `localhost` when left out), `name` for `process` (the program's file name; an optional `command_has` also needs a piece of its text, ignoring case: the full executable path on Windows (as the system names it, after the drive: \Device\HarddiskVolumeN\..., so match from the folder, not the drive letter), the whole command line on Linux and macOS, so `command_has = '\envs\imageresizer\'` tells one app's `python.exe` from the rest, a backslash and a slash count the same, and a process that cannot be inspected does not match; an optional `at_least = 30` is the fewest matching processes that is healthy: fewer is `degraded`, none is `down`, and the value reported is how many were found, which is how an app that runs as dozens of worker processes is watched; see Which processes can be seen), `windows-service` and `systemd` (the unit), `metric` and `limit` for `threshold` (bad when the metric is over the limit), `date_from` for `expiry` (an `https` address or `host:port` for a certificate, or a file whose first line is a date; the check is `degraded` from 30 days before the date, counted in exact time, and `down` once it has passed, and shows the days left rounded up). All take `every` and `fail_after`, the number of bad results in a row (a whole number from 1 to 100, 3 when left out) after which the server opens an issue. `http` also takes `slow`: an answer that comes back successfully but takes longer is `degraded`, with the time as its value, and the server opens an issue only after its usual run of bad results. | | `[stats]` | Where the stats document is and which environment variable holds its key. | | `[logs]` | Files to read. Paths are relative to the file and must stay inside its folder. Each line is a JSON object (`t`, `level`, `msg`, `service`, `fields`) or a CLEF line; any other line is kept as its message. | | `[[job]]` | A scheduled job: when it should run, how late it may be (`grace`, 30 minutes when left out), and how long it may take (`max_runtime`, 1 hour when left out). `schedule` is a five-field cron line read as UTC, and `@reboot` is refused. | | `[alert]` | A starting suggestion for who is told. | ## Rules **The file owns what the app is and how to check it. The interface owns who is told, snoozes and overrides.** The interface shows a file-declared check as coming from the file, and it is changed by changing the file. `[alert]` only seeds the routing the first time the service appears. **Secrets never go in the file.** A field ending in `_env` names an environment variable that holds one. **A declared check is active straight away.** ## How the file reaches Dokkit Two ways, both supported: 1. **`dokkit` reads it.** `dokkit`'s own configuration lists the folders it may read `dokkit.toml` files from. That configuration is only editable by an administrator. `dokkit` re-reads a file when it changes. 2. **It is sent.** `dokkit apply` sends the file from anywhere that has a key, for example from CI. A machine key may declare any service on its machine. An app key may declare only the service it belongs to: `[service] id` must match, the machine and kind stay as they are, `[service]` may set the name and environment, `[[job]]` keeps `id`, `schedule`, `grace` and `max_runtime`, `[[check]]` may be of type `beat` or `custom` only (any other type is refused), and the other sections are dropped. ## Which processes can be seen A `process` check sees the processes of every account, with no administrator rights, on each system, and matches by the program's file name (and `command_has` against the text below). - **Windows.** The list of processes comes from a system snapshot, which includes other accounts' processes. The executable path used by `command_has` is asked of the system by process number, without opening the process, so Local Service sees other accounts' processes and their paths (`procs_windows.go`). A protected system process may give no path; it then matches by name only. - **Linux.** `/proc//comm` and `/proc//cmdline` are readable by any user, so an unprivileged account sees all processes and their command lines. Where `/proc` is mounted with `hidepid=1` or `hidepid=2`, an account outside the allowed group sees only its own, and a process that is not listed is counted as not running. The program name is the kernel's, cut to 15 characters. The executable path (`/proc//exe`) is not used, because only the owner or root can read it. - **macOS.** `ps -axww` lists every user's processes with their command lines for any account, so the same applies as on Linux without `hidepid`. ## Checking it `dokkit check` validates the file, runs every check once on the current machine, and prints exactly what would be sent. It needs no server. It refuses what the server refuses when the file is declared: `service.id` outside lower-case letters, digits, dots, dashes and underscores (starting with a letter or digit), a `service.kind` that is not `service`, `job`, `dependency` or `app`, a check type that is not known, a job without an `id`, a job `schedule` that is not a cron line (five fields read as UTC, or `@hourly`, `@daily`, `@weekly`, `@monthly`, `@yearly`; `@reboot` is refused), and a `fail_after` that is not a whole number from 1 to 100. It also refuses a `key_env` that does not start with `DOKKIT_STATS_`, a `[stats] url` that is not an `http` or `https` address, and a `[logs] path` that leaves the file's folder. `dokkit watch` keeps a file with one of those last three and reports the part that cannot work. `spec/declare-cases.json` holds files both the program and the server's declare are tested against. # The `dokkit` command line One program on each watched machine, also used in an app's repo. `dokkit help` prints this list and `dokkit help ` says more about one. ## In an app's repo ### `dokkit init` Asks what the app is and writes dokkit.toml. Asks a few questions and writes a dokkit.toml in this folder: what the app is, and how to tell it is healthy. Nothing is sent. ### `dokkit check [dokkit.toml]` Runs the checks it can once and prints what would be sent. Checks the file's schema, service id and each check's fields. Runs the http, tcp, process, windows-service, systemd and expiry checks that can run here, once, and prints exactly what would be sent. Checks the app reports itself are listed as not run. A check aimed at another host does make its request. Needs no server and sends nothing to it. A slow answer shows as slow, not as a failure. ### `dokkit apply [dokkit.toml]` Sends the file to the server. Declares the file to the server, for a file dokkit watch does not read. It sends DOKKIT_KEY when that is set, otherwise this machine's key from dokkit enrol. A machine key may declare any service on its machine; an app key only its own service. Sending the same file twice changes nothing. ## Jobs ### `dokkit run [--service ] --job -- [arguments]` Runs a command and reports how it went. Runs the command and reports its start, finish, exit code, duration and last lines of output. Exits with the command's own exit code. With this machine's key, a job declared in a dokkit.toml in the folders of the saved configuration is found by name, and --service says which service when it is not. With an app key, --service is ignored. ### `dokkit ping [--service ] [start|ok|fail]` Reports a job that cannot be wrapped. Reports one event for a job: start, ok (the default) or fail. ## This machine ### `dokkit enrol
[sha256:]` Joins this machine to a server. Trades a one-time token for this machine's key and saves it. Given a fingerprint, it trusts the certificate with that fingerprint and no other. ### `dokkit status` What this machine watches and when it last reported. Shows the files dokkit watch reads, when it last sent, and how many reports are waiting, and any folder it cannot read. It says a watch is running only while that watch has written down in the last three minutes, and then gives its process number. On Linux it shows the installed service when root, or the dokkit group, can read its files; the service's status itself can be read only with sudo. ### `dokkit quiet [reason]` Tells the server this machine is going down on purpose. Sends what is waiting in the spool, then tells the server this machine will be down for the length given (such as 90s, 15m or 4h; at most 4h), with an optional reason. For that long the server opens no issue and sends nothing for the machine going silent or its checks failing; when the time is up, or the machine has reported again and each of its checks has, the usual rules resume, and a machine still silent then is reported as stopped. It prints one line and exits 0 once the server has answered, and prints why and exits 1 when it could not be told, so a script can decide whether to go ahead. A second call replaces the window. Uses this machine's key. ### `dokkit quiet end` Ends the planned downtime now. Tells the server the wait is over: the usual rules apply from this moment. ### `dokkit doctor` Checks the server, the key and the clock, and says what to do. Says whether the server answers and knows this key, whether this machine's clock is within 30 seconds of the server's (a slow answer counts as clock difference, up to 10 seconds), and whether reports are waiting. Each way it can fail reads differently and ends with the one thing to do: a name that does not resolve, a port nothing listens on, no answer in time, a certificate that is not the one saved at enrol (a pin mismatch), one no authority signed, a key the server does not know, a clock off by over 30 seconds. On a Mac whose background copy runs but cannot send to a server on the local network while this terminal can, it says macOS's local network privacy is the likely cause and what to allow. ## Finding things ### `dokkit scan [--no-services] [--no-ports] [--no-containers] [--no-sites] [--no-disks] [--no-jobs] [--no-declarations]` Looks over this machine and writes a proposal file. Lists what is on the machine that could be watched and writes it to dokkit-proposal.json in this folder. Sends nothing. Each --no- flag leaves one kind of thing out. Jobs are scheduled work: Task Scheduler tasks on Windows, cron entries and systemd timers on Linux, cron entries and launchd jobs on macOS. Each is listed with its schedule, last run and command line, to be watched, never run. A scan uses the machine's own listings, takes a few seconds and changes nothing. dokkit watch does the same on its own shortly after it starts and once a day, and when the Found page asks it to look again. ### `dokkit propose` Sends the proposal file to be accepted. Sends dokkit-proposal.json, which a scan wrote. Nothing in it is watched until a person accepts it in the interface, or a token allowed to act accepts it through the API. ## Staying running ### `dokkit watch [--background]` Watches this machine, as an ordinary program. Reads dokkit.toml files from the folders in DOKKIT_FOLDERS or in the saved configuration, and from the folders directly inside them. Runs their checks, sends the log files they name, and says the machine is alive once a minute. No administrator, no service. --background leaves it running after this window closes. Run like this, it never restarts a service or unit; only the installed service does. ## Finding things ### `dokkit import seq --from [--seq-key ] [--to
] [--key ] [--default-service ] [--map ] [--service-property ] [--app-pattern ] [--state ] [--since