domino_admin_toolkit.checks.test_backup_cronjobs module

Description:

Domino runs a regular backup of some of it’s core services on a cron schedule These are important to recover from certain issues in a DR situation.

pydantic model domino_admin_toolkit.checks.test_backup_cronjobs.BackupCronjobAnalyzer

Bases: AnalyzerBase[BackupCronjobRow]

Fields:

analyze(data)

Analyzes one row and returns a list of CheckResult instances.

Return type:

list[CheckResult]

Args:

data: One row dict (TRow). The Runner calls this once per DataFrame row.

Returns:

List[CheckResult]: A list containing the results of the analysis.

Raises:

NotImplementedError: If this method is not implemented by subclasses.

name: ClassVar[str] = 'BackupCronjobAnalyzer'
class domino_admin_toolkit.checks.test_backup_cronjobs.BackupCronjobRow

Bases: TypedDict

cronjob_name: str
has_successful_backup: bool
hours_since_last_success: float | None
last_successful_completion_time: datetime | None
last_successful_job_name: str | None
domino_admin_toolkit.checks.test_backup_cronjobs.backup_cronjob_data(platform_namespace, domino_version, k8s_client)

Collect backup cronjob data - finds last successful backup for each cronjob.

domino_admin_toolkit.checks.test_backup_cronjobs.test_backup_cronjob_status(backup_cronjob_data, runner)

Description: Validates that backup cronjobs have completed successfully within the last 24 hours. This check verifies the time since the last successful backup rather than checking for failed jobs, as failed backup pods can remain forever due to how K8s cron jobs work.

Failure Conditions:

  • Last successful backup was more than 24 hours ago

  • No successful backups found for any backup cronjob

  • No backup cronjobs found in deployment

Success Conditions:

  • At least one successful backup completed within the last 24 hours for each backup cronjob

Troubleshooting Steps:

  1. Check cronjob status via kubectl get cronjobs -n <platform-namespace>

  2. Review job logs via kubectl logs -n <platform-namespace> <job-name>

  3. Verify backup storage availability and permissions

  4. Check if backup jobs are running on schedule

Resolution Steps:

  1. Manually trigger backup job via kubectl create job –from=cronjob/<cronjob-name>

  2. Check backup storage configuration in Domino platform

  3. Verify RBAC permissions for backup service account

  4. Review cronjob schedule configuration

Required Permissions: Platform admin access to view jobs and cronjobs, backup storage admin

Note: Failed backup pods are excluded from pod health checks and handled by this backup-specific check.