hardis:org:monitor:backup

Description

Retrieve sfdx sources in the context of a monitoring backup

The command exists in 2 modes: filtered(default & recommended) and full.

Filtered mode (default, better performances)

Automatically skips metadatas from installed packages with namespace.

You can remove more metadata types from backup, especially in case you have too many metadatas and that provokes a crash, using:

• Manual update of manifest/package-skip-items.xml config file (then commit & push in the same branch)

  • Works with full wildcard (<members>*</members>) , named metadata (<members>Account.Name</members>) or partial wildcards names (<members>pi__*</members> , <members>*__dlm</members> , or <members>prefix*suffix</members>)

• Environment variable MONITORING_BACKUP_SKIP_METADATA_TYPES (example: MONITORING_BACKUP_SKIP_METADATA_TYPES=CustomLabel,StaticResource,Translation): that will be applied to all monitoring branches.

Data Cloud objects backup

When the org contains Data Cloud objects (__dlm and __dll suffix on CustomObjects), they are backed up by default in a separate retrieve call.

To skip Data Cloud objects backup, use config property monitoringBackupSkipDataCloud: true or environment variable MONITORING_BACKUP_SKIP_DATA_CLOUD=true.

Complete Profiles

A file-based retrieve returns a Profile with only the permissions related to the other components of the same request, so a backed up Profile used to miss most of what it grants (standard objects and their fields, tabs, and in full mode everything that is in another chunk).

After the retrieve, the Profiles are read again with the CRUD Metadata API, which returns them whole, and written over the retrieved files. This is what the Auto mode of the VS Code Metadata Retriever does, and what sf hardis:mdapi:read --metadata Profile --active-only does.

• The entries that grant nothing are left out (field access with no read and no edit, Apex class access disabled, user permission disabled...), and so are the tab settings of tabs that do not exist. A permission that is revoked in the org shows as a removed line in the next backup.
• Only the Profiles that were retrieved are read: a Profile skipped with manifest/package-skip-items.xml or MONITORING_BACKUP_SKIP_METADATA_TYPES stays skipped.
• A Profile that can not be read fails the backup, with the names of the Profiles and the error. Skip it with manifest/package-skip-items.xml to let the backup go on.
• The first backup that runs with this behavior reports every Profile as updated.

Other types, including Permission Sets, come back whole from the file-based retrieve and are not read again.

Deleted components

A retrieve never deletes a local file. After the retrieve, the backup deletes the files of the components that no longer exist in the org, so a component deleted in the org shows as deleted files in the next backup.

• A file is deleted only when its component is not in manifest/package-all-org-items.xml, the list of everything that exists in the org. A component that still exists but is no longer retrieved (manifest/package-skip-items.xml, MONITORING_BACKUP_SKIP_METADATA_TYPES, installed package namespace) keeps the file of its last backup.
• Before deleting, the backup asks the org for each component with the Metadata API, and deletes its files only when the org does not return it. A component the org still returns stays, and so does one the org can not be asked about (a type the Metadata API does not read, like Settings). The files kept this way are listed in the log.
• A file that the org just returned is never deleted, and neither is a file that can not be linked to a component.
• Some files stay when nothing proves that their component is gone: the children (fields, list views, record types...) of an object that is not retrieved, the types that the org does not list by name, and the types of which nothing is retrieved (a type skipped as a whole, like Report). For those, only the components that the previous backup listed and that the org no longer lists are deleted: the list of the org alone is not a proof, it misses components that exist, such as the reports of nested folders.
• The files of GlobalValueSetTranslation and StandardValueSetTranslation are never deleted: the org lists one language of a value set and not the others.
• There is no limit on the number of deleted files: a cleanup that removes a lot of metadata is mirrored by the next backup.
• An error while deleting the files is logged as a warning: the backup goes on.
• The first backup that runs with this behavior deletes the files of everything that was deleted in the org since the monitoring started.
• Nothing is deleted when the run does not retrieve everything (--start-chunk, FULL_ORG_MANIFEST_PATH).

To keep the files of deleted components, use config property monitoringBackupSkipDeletions: true or environment variable MONITORING_BACKUP_SKIP_DELETIONS=true.

Full mode

Activate it with --full parameter, or variable MONITORING_BACKUP_MODE_FULL=true

Ignores filters (namespaces items & manifest/package-skip-items.xml) to retrieve ALL metadatas, including those you might not care about (reports, translations...)

As we can retrieve only 10000 files by call, the list of all metadatas will be chunked to make multiple calls (and take more time than filtered mode)

• if you use --full-apply-filters , manifest/package-skip-items.xml and MONITORING_BACKUP_SKIP_METADATA_TYPES filters will be applied anyway
• if you use --exclude-namespaces , namespaced items will be ignored

With those both options, it's like if you are not using --full, but with chunked metadata download

In CI/CD

This command is part of sfdx-hardis Monitoring and can output Grafana, Slack and MsTeams Notifications. When the backup fails, it sends a BACKUP notification with the error severity and the error message, then exits with that error.

Coding agents

After each backup, the command writes an AGENTS.md file at the root of the repository. It explains to a coding agent (Claude Code, Codex, Gemini, Copilot...) how the monitoring works, what each file and folder holds, what is not backed up, how to use the git history to answer questions about the org, and which monitoring checks are configured on the branch.

Only the block between the sfdx-hardis-monitoring-agents-start and sfdx-hardis-monitoring-agents-end markers is rewritten: notes written after the end marker are kept. When the markers are broken (one of them deleted, or several pairs), the file is left untouched and the command logs a warning. A CLAUDE.md file that imports AGENTS.md is also created when the repository has none.

MegaLinter

The MegaLinter job of a monitoring pipeline reports its findings and must not fail because of them. After each backup, the command adds DISABLE_ERRORS: true to the .mega-linter.yml of the repository when the key is missing, and creates the file when the repository has none. A DISABLE_ERRORS key that is already there is never changed: write DISABLE_ERRORS: false to make the job fail on linter errors.

Troubleshooting

If you have unknown errors (it happens !), you can investigate using the full command with smaller chunks.

Example: sf hardis:org:monitor:backup --full --exclude-namespaces --full-apply-filters --max-by-chunk 500

It will allow you the identify the responsible metadata and ignore it using package-skip-items.xml or MONITORING_BACKUP_SKIP_METADATA_TYPES env variable.

Documentation

Doc generation (including visual flows) is triggered at the end of the command.

If you want to also upload HTML Documentation on your Salesforce Org as static resource, use variable SFDX_HARDIS_DOC_DEPLOY_TO_ORG="true"

If you want to also upload HTML Documentation on Cloudflare, use variable SFDX_HARDIS_DOC_DEPLOY_TO_CLOUDFLARE="true"

If you want to also publish Documentation on Confluence, use variable SFDX_HARDIS_DOC_DEPLOY_TO_CONFLUENCE="true" (see configuration)

By default, only changed files are used to generate the documentation (diff-only). If you want to force the full documentation rebuild, use option --rebuild-full-doc or set env variable MONITORING_BACKUP_REBUILD_FULL_DOC="true"

• If you want to generate the documentation in multiple languages, define variable SFDX_DOC_LANGUAGES (ex: SFDX_DOC_LANGUAGES=en,fr,de)
• You can define one Cloudflare site by language, for example with the following variables:
  • CLOUDFLARE_PROJECT_NAME_EN=cloudity-demo-english
  • CLOUDFLARE_PROJECT_NAME_FR=cloudity-demo-french
  • CLOUDFLARE_PROJECT_NAME_DE=cloudity-demo-german

If Flow history doc always display a single state, you probably need to update your workflow configuration:

• on Gitlab: Env variable GIT_FETCH_EXTRA_FLAGS: --depth 10000
• on GitHub: fetch-depth: 0
• on Azure: fetchDepth: "0"
• on Bitbucket: step: clone: depth: full

Agent Mode

Supports non-interactive execution with --agent:

sf hardis:org:monitor:backup --agent --target-org myorg@example.com

In agent mode:

• Documentation generation proceeds without prompting.
• The rebuild mode defaults to diff-only (changed files only) unless --rebuild-full-doc or MONITORING_BACKUP_REBUILD_FULL_DOC=true is set.
• All other interactive prompts are skipped.

Parameters

Examples

$ sf hardis:org:monitor:backup

$ sf hardis:org:monitor:backup --full

$ sf hardis:org:monitor:backup --full --exclude-namespaces

$ sf hardis:org:monitor:backup --full --exclude-namespaces --full-apply-filters

$ sf hardis:org:monitor:backup --rebuild-full-doc

$ sf hardis:org:monitor:backup --agent

$ FULL_ORG_MANIFEST_PATH=manifest/package-all-org-items.xml sf hardis:org:monitor:backup

Comments