opct report
opct report generates a comprehensive report from the conformance workflow executed on an OpenShift/OKD cluster.
The report is available in two formats:
- Command Line Interface (CLI)
- Web User Interface (Web UI)
The report command is the most important command used to review the results.
The command reads the archive/result file and: - provides a summary of standard conformance suites required to validate an OpenShift/OKD installation - extracts must-gather information, such as the state of objects and logs - extracts counters of error patterns from workloads (must-gather) - applies SLOs (Checks) created from results executed in well-known (reference) deployments - reports failed SLOs, providing guidance on failed items to help/guide the OpenShift/OKD installation - exports counters and exposes them to be used as indicators when reviewing an OpenShift/OKD installation - builds a local Web UI app allowing you to navigate throughout the results extracted from the archive, such as e2e test failure logs, e2e test metadata, e2e test documentation, CAMGI report, metrics report, and counters
To begin, see the Usage section.
Usage
-
Basic usage:
opct report <archive.tar.gz> -
Advanced usage exposing/serving the WebUI:
opct report <archive.tar.gz> --save-to /tmp/results --log-level=debug
Options
Create a report from results.
Usage:
opct report archive.tar.gz [flags]
Flags:
-b, --baseline string [DEPRECATED] Baseline result archive file. Example: -b file.tar.gz
-d, --diff string [DEPRECATED] Diff results from a baseline archive file. Example: --diff file.tar.gz
--embed-data Force to embed the data into HTML report, allwoing the use of file protocol/CORS in the browser.
-f, --force Force to continue the execution, skipping deprecation warnings.
-h, --help help for report
--json Show report in json format
--save-only Save data and exit. Requires --save-to. Example: -s ./results --save-only
-s, --save-to string Extract and Save Results to disk. Example: -s ./results
--server-address string HTTP server address to serve files when --save-to is used. Example: --server-address 0.0.0.0:9090 (default "0.0.0.0:9090")
--skip-baseline-api Set to disable the BsaelineAPI call to get the baseline results injected in the failure filter pipeline.
--skip-server HTTP server address to serve files when --save-to is used. Example: --server-address 0.0.0.0:9090
-v, --verbose Show test details of test failures
Global Flags:
--kubeconfig string kubeconfig for target OpenShift cluster
--log-level string logging level (default "info")
Examples
CLI Report
To generate a CLI report, run the following command:
./opct report ./opct_202502230112_624d66d9-6354-40d1-b847-82a55f57d444.tar
The output will be similar to:
time="2025-02-26T14:00:29-03:00" level=info msg="Creating report..."
time="2025-02-26T14:00:31-03:00" level=info msg="Processing Plugin 10-openshift-kube-conformance..."
time="2025-02-26T14:00:31-03:00" level=info msg="Processing Plugin 20-openshift-conformance-validated..."
time="2025-02-26T14:00:32-03:00" level=info msg="Processing Plugin 80-openshift-tests-replay..."
time="2025-02-26T14:00:32-03:00" level=info msg="Processing Plugin 99-openshift-artifacts-collector..."
time="2025-02-26T14:00:32-03:00" level=info msg="Processing Plugin 05-openshift-cluster-upgrade..."
time="2025-02-26T14:00:32-03:00" level=info msg="Processing results..."
┌───────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ OPCT Summary │
│ > Archive: opct_202410270102_5c274438-ea74-443f-ac4a-d28effdc042c.tar │
├─────────────────────────────────────┬─────────────────────────────────────────────────────────────────────────────┤
│ │ PROVIDER │
├─────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ Infrastructure: │ │
│ PlatformType │ External (aws) │
│ Name │ ci-op-dvv014fp-33e9d-4wt5w │
│ ClusterID │ 0f206ba4-452c-4f8d-b2a8-efb98b65d47c │
│ Topology │ HighlyAvailable │
│ ControlPlaneTopology │ HighlyAvailable │
│ API Server URL │ https://api.ci-op-dvv014fp-33e9d.origin-ci-int-aws.dev.rhcloud.com:6443 │
│ API Server URL (internal) │ https://api-int.ci-op-dvv014fp-33e9d.origin-ci-int-aws.dev.rhcloud.com:6443 │
│ NetworkType │ OVNKubernetes │
├─────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ Cluster Version: │ │
│ Kubernetes │ v1.31.2 │
│ OpenShift │ 4.18.0-0.nightly-2024-10-26-063703 │
│ Channel │ stable-4.18 │
├─────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ Cluster Status: │ Available │
│ Cluster Status/Conditions: │ │
│ Available │ True │
│ Failing │ False │
│ Progressing (Update) │ False │
│ RetrievedUpdates │ False │
│ EnabledCapabilities │ False │
│ ReleaseAccepted │ True │
├─────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ Plugin summary: │ Status [Total/Passed/Failed/Skipped] (timeout) │
│ 10-openshift-kube-conformance │ passed [402/402/0/0] (0) │
│ 20-openshift-conformance-validated │ failed [3871/1581/21/2269] (0) │
│ 05-openshift-cluster-upgrade │ passed [1/0/0/1] (0) │
├─────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ Env health summary: │ [A=True/P=True/D=True] │
│ Cluster Operators │ [33/0/0] │
│ Node health │ 6/6 (100.00%) │
│ Pods health │ 277/277 (100.00%) │
├─────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ Test count by suite: │ │
│ kubernetes/conformance │ 415 │
│ openshift/conformance │ 3885 │
└─────────────────────────────────────┴─────────────────────────────────────────────────────────────────────────────┘
=> Processed Summary <=
==> Result Summary by test suite:
┌───────────────────────────────────────────┐
│ 05-openshift-cluster-upgrade: ✅ │
├───────────────────────────┬───────────────┤
│ Total tests │ 1 │
│ Passed │ 0 │
│ Failed │ 0 │
│ Timeout │ 0 │
│ Skipped │ 1 │
│ Result Job │ passed │
└───────────────────────────┴───────────────┘
┌───────────────────────────────────────────┐
│ 10-openshift-kube-conformance: ✅ │
├───────────────────────────┬───────────────┤
│ Total tests │ 402 │
│ Passed │ 402 │
│ Failed │ 0 │
│ Timeout │ 0 │
│ Skipped │ 0 │
│ Filter Failed Suite │ 0 (0.00%) │
│ Filter Failed KF │ 0 (0.00%) │
│ Filter Replay │ 0 (0.00%) │
│ Filter Failed Baseline │ 0 (0.00%) │
│ Filter Failed Priority │ 0 (0.00%) │
│ Filter Failed API │ 0 (0.00%) │
│ Failures (Priotity) │ 0 (0.00%) │
│ Result - Job │ passed │
│ Result - Processed │ passed │
└───────────────────────────┴───────────────┘
┌───────────────────────────────────────────┐
│ 20-openshift-conformance-validated: ❌ │
├───────────────────────────┬───────────────┤
│ Total tests │ 3871 │
│ Passed │ 1581 │
│ Failed │ 21 │
│ Timeout │ 0 │
│ Skipped │ 2269 │
│ Filter Failed Suite │ 18 (0.46%) │
│ Filter Failed KF │ 18 (0.46%) │
│ Filter Replay │ 18 (0.46%) │
│ Filter Failed Baseline │ 18 (0.46%) │
│ Filter Failed Priority │ 18 (0.46%) │
│ Filter Failed API │ 7 (0.18%) │
│ Failures (Priotity) │ 7 (0.18%) │
│ Result - Job │ failed │
│ Result - Processed │ failed │
└───────────────────────────┴───────────────┘
┌───────────────────────────────────────────┐
│ 80-openshift-tests-replay: ✅ │
├───────────────────────────┬───────────────┤
│ Total tests │ 1 │
│ Passed │ 0 │
│ Failed │ 0 │
│ Timeout │ 0 │
│ Skipped │ 1 │
│ Filter Failed Suite │ 0 (0.00%) │
│ Filter Failed KF │ 0 (0.00%) │
│ Filter Replay │ 0 (0.00%) │
│ Filter Failed Baseline │ 0 (0.00%) │
│ Filter Failed Priority │ 0 (0.00%) │
│ Filter Failed API │ 0 (0.00%) │
│ Failures (Priotity) │ 0 (0.00%) │
│ Result - Job │ passed │
│ Result - Processed │ passed │
└───────────────────────────┴───────────────┘
┌───────────────────────────────────────────┐
│ 99-openshift-artifacts-collector: ✅ │
├───────────────────────────┬───────────────┤
│ Total tests │ 16 │
│ Passed │ 16 │
│ Failed │ 0 │
│ Timeout │ 0 │
│ Skipped │ 0 │
│ Result Job │ passed │
└───────────────────────────┴───────────────┘
==> Result details by conformance plugins:
==> 10-openshift-kube-conformance - test failures:
==> 20-openshift-conformance-validated - test failures:
┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ ==> 20-openshift-conformance-validated │
│ 🚨 ACTION REQUIRED: Failed tests to review │
├──────┬──────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ #ERR │ TEST NAME │
├──────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ 13 │ [sig-network][Feature:Router] The HAProxy router should enable openshift-monitoring to pull metrics │
│ │ [Skipped:Disconnected] [Suite:openshift/conformance/parallel] │
├──────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ 7 │ [sig-api-machinery][Feature:ResourceQuota] Object count check the quota after import-image with --al │
│ │ l option [Skipped:Disconnected] [Suite:openshift/conformance/parallel] │
├──────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ 7 │ [sig-network][Feature:Router] The HAProxy router should expose a health check on the metrics port [S │
│ │ kipped:Disconnected] [Suite:openshift/conformance/parallel] │
├──────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ 7 │ [sig-network][Feature:Router] The HAProxy router should expose prometheus metrics for a route [apigr │
│ │ oup:route.openshift.io] [Skipped:Disconnected] [Suite:openshift/conformance/parallel] │
├──────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ 7 │ [sig-network][Feature:Router] The HAProxy router should expose the profiling endpoints [Skipped:Disc │
│ │ onnected] [Suite:openshift/conformance/parallel] │
├──────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ 3 │ [sig-network][Feature:tap] should create a pod with a tap interface [apigroup:k8s.cni.cncf.io] [Suit │
│ │ e:openshift/conformance/parallel] │
├──────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ 1 │ [sig-arch] Managed cluster should set requests but not limits [Suite:openshift/conformance/parallel] │
├──────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ │ [TOTAL=7] [SIG-NETWORK=5 (71.43%)] [SIG-API-MACHINERY=1 (14.29%)] [SIG-ARCH=1 (14.29%)] │
└──────┴──────────────────────────────────────────────────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Validation checks / Results │
├───────────┬───┬────────┬────────────────────────────────────────────────────────────────────────────────────────┬──────────────────────────────┬─────────────────┤
│ ID │ # │ RESULT │ CHECK NAME │ TARGET │ CURRENT │
├───────────┼───┼────────┼────────────────────────────────────────────────────────────────────────────────────────┼──────────────────────────────┼─────────────────┤
│ OPCT-005B │ ⚠ │ warn │ OpenShift Conformance Validation [20]: Required to Pass After Filters │ Pass==100%(W<=0.50%,F>0.50%) │ Fail==0.18%(7) │
│ OPCT-010 │ ⚠ │ warn │ The cluster logs generates accepted error budget │ W:<=30k,F:>100k │ 46977 │
├───────────┼───┼────────┼────────────────────────────────────────────────────────────────────────────────────────┼──────────────────────────────┼─────────────────┤
│ OPCT-020 │ ✔ │ pass │ All nodes must be healthy │ 100% │ 100.000% │
│ OPCT-021 │ ✔ │ pass │ Pods Healthy must report higher than 98% │ >=98% │ 100.000 │
│ OPCT-001 │ ✔ │ pass │ Kubernetes Conformance [10-openshift-kube-conformance] must pass 100% │ Priority==0|Total!=Failed │ Priority==0 │
│ OPCT-004 │ ✔ │ pass │ OpenShift Conformance [20-openshift-conformance-validated]: Pass ratio must be >=98.5% │ Pass>=98.5%(Fail>1.5%) │ Fail==0.54%(21) │
│ OPCT-005 │ ✔ │ pass │ OpenShift Conformance Validation [20]: Filter Priority Requirement >= 99.5% │ W<=0.50%,F>0.50% │ Fail==0.46%(18) │
│ OPCT-011 │ ✔ │ pass │ The test suite generates accepted error budget │ Pass<=150(W>150,F>300) │ 128 │
│ OPCT-003 │ ✔ │ pass │ Plugin Collector [99-openshift-artifacts-collector] must pass │ passed │ passed │
│ OPCT-002 │ ✔ │ pass │ Plugin Conformance Upgrade [05-openshift-cluster-upgrade] must pass │ passed │ passed │
│ OPCT-010A │ ✔ │ pass │ etcd logs: slow requests: average should be under 500ms │ <=500.00 ms │ 414.493 │
│ OPCT-010B │ ✔ │ pass │ etcd logs: slow requests: maximum should be under 1000ms │ <=1000.00 ms │ 638.876 │
│ OPCT-022 │ ✔ │ pass │ Detected one or more plugin(s) with potential invalid result │ passed │ passed │
│ OPCT-023A │ ✔ │ pass │ Sanity [10-openshift-kube-conformance]: potential missing tests in suite │ F:<300 │ Total==402 │
│ OPCT-023B │ ✔ │ pass │ Sanity [20-openshift-conformance-validated]: potential missing tests in suite │ F:<3000 │ Total==3871 │
│ OPCT-030 │ ✔ │ pass │ Node Topology: ControlPlaneTopology HighlyAvailable must use multi-zone │ W:>1,P:>2 │ Zones==2 │
│ -- │ ✔ │ pass │ Platform Type must be supported by OPCT │ None|External|AWS|Azure │ External │
│ -- │ ✔ │ pass │ Cluster Version Operator must be Available │ True │ True │
│ -- │ ✔ │ pass │ Cluster condition Failing must be False │ False │ False │
│ -- │ ✔ │ pass │ Cluster upgrade must not be Progressing │ False │ False │
│ -- │ ✔ │ pass │ Cluster ReleaseAccepted must be True │ True │ True │
│ -- │ ✔ │ pass │ Infrastructure status must have Topology=HighlyAvailable │ HighlyAvailable │ HighlyAvailable │
│ -- │ ✔ │ pass │ Infrastructure status must have ControlPlaneTopology=HighlyAvailable │ HighlyAvailable │ HighlyAvailable │
├───────────┼───┼────────┼────────────────────────────────────────────────────────────────────────────────────────┼──────────────────────────────┼─────────────────┤
│ │ │ │ TOTAL: 23, FAILED: 0 (0.00%), WARN: 2 (8.70%), PASS: 21 (91.30%), SKIP: 0 (0.00%) │ │ │
└───────────┴───┴────────┴────────────────────────────────────────────────────────────────────────────────────────┴──────────────────────────────┴─────────────────┘
Web UI Report
To generate a Web UI report, follow these steps:
- Run the command:
./opct report --save-to ./results ./opct_202502230112_624d66d9-6354-40d1-b847-82a55f57d444.tar
- Open your browser and navigate to localhost:9090.
