2022-10-14 14:08:11,165 - xtesting.ci.run_tests - INFO - Deployment description: +-------------------------+------------------------------------------------------------+ | ENV VAR | VALUE | +-------------------------+------------------------------------------------------------+ | CI_LOOP | daily | | DEBUG | false | | DEPLOY_SCENARIO | k8-nosdn-nofeature-noha | | INSTALLER_TYPE | unknown | | BUILD_TAG | 0M9L4S6Y5T7C | | NODE_NAME | lf-virtual1-8 | | TEST_DB_URL | http://testresults.opnfv.org/test/api/v1/results | | TEST_DB_EXT_URL | http://testresults.opnfv.org/test/api/v1/results | | S3_ENDPOINT_URL | https://storage.googleapis.com | | S3_DST_URL | s3://artifacts.opnfv.org/functest- | | | kubernetes/0M9L4S6Y5T7C/functest-kubernetes-opnfv- | | | functest-kubernetes-cnf-v1.25-cnf_testsuite-run-9 | | HTTP_DST_URL | http://artifacts.opnfv.org/functest- | | | kubernetes/0M9L4S6Y5T7C/functest-kubernetes-opnfv- | | | functest-kubernetes-cnf-v1.25-cnf_testsuite-run-9 | +-------------------------+------------------------------------------------------------+ 2022-10-14 14:08:11,180 - xtesting.ci.run_tests - INFO - Loading test case 'cnf_testsuite'... 2022-10-14 14:08:11,240 - xtesting.ci.run_tests - INFO - Running test case 'cnf_testsuite'... 2022-10-14 14:08:13,114 - functest_kubernetes.cnf_conformance.conformance - INFO - cnf-testsuite setup Successfully created directories for cnf-testsuite Global helm found. Version: v3.3.1 Local helm found. Version: v3.3.1 Global kubectl found. Version: 1.25 No Local kubectl version found Global git found. Version: 2.34.4 No Local git version found All prerequisites found. Setup complete 2022-10-14 14:09:03,460 - functest_kubernetes.cnf_conformance.conformance - INFO - cnf-testsuite cnf_setup cnf-config=cnf-testsuite.yml Successfully created directories for cnf-testsuite cnf setup online mode Helm pull error Successfully setup coredns cnf setup online mode complete 2022-10-14 14:34:30,677 - functest_kubernetes.cnf_conformance.conformance - INFO - cnf-testsuite workload ✔️ PASSED: hostPath volumes not found 🖥️ 💾 State final score: 5 of 10  ✔️ PASSED: No privileged containers 🔓🔑 ✔️ PASSED: Root user not found 🚫√ Security final score: 10 of 10  Warning: spec.template.metadata.annotations[scheduler.alpha.kubernetes.io/critical-pod]: non-functional in v1.16+; use the "priorityClassName" field instead ✔️ PASSED: Replicas increased to 3 📦📈 Warning: spec.template.metadata.annotations[scheduler.alpha.kubernetes.io/critical-pod]: non-functional in v1.16+; use the "priorityClassName" field instead ✔️ PASSED: Replicas decreased to 1 📦📉 Scalability final score: 20 of 35  ✔️ PASSED: No IP addresses found ✔️ PASSED: Helm liveness probe found ⎈🧫 ✔️ PASSED: Helm readiness probe found ⎈🧫 ✔️ PASSED: NodePort is not used ✔️ PASSED: No hard-coded IP addresses found in the runtime K8s configuration Warning: spec.template.metadata.annotations[scheduler.alpha.kubernetes.io/critical-pod]: non-functional in v1.16+; use the "priorityClassName" field instead ✔️ PASSED: CNF Rollback Passed No Secret Volumes or Container secretKeyRefs found for resource: {kind: "Deployment", name: "coredns-coredns"} ⏭ SKIPPED: Secrets not used 🧫 To address this issue please see the USAGE.md documentation  ✖️ FAILED: Found mutable configmap(s) ⚖️ ✔️ PASSED: CNF for Rolling Update Passed ✔️ PASSED: CNF for Rolling Downgrade Passed ✔️ PASSED: CNF for Rolling Version Change Passed Configuration Lifecycle final score: 40 of 46  ✔️ PASSED (by default): No install script provided Successfully created directories for cnf-testsuite ✔️ PASSED: Helm Chart exported_chart Lint Passed ⎈📝☑️ ✔️ PASSED: Published Helm Chart Found ⎈📦🌐 SKIPPED: Helm Deploy Installability final score: 15 of 20  ✔️ PASSED: Image size is good 🐜 ⚖️👀 Pod Ready Status: true ✖️ FAILED: CNF had a startup time of 49 seconds 🐢 ✔️ PASSED: Only one process type used 🐜 ⚖️👀 Microservice final score: 9 of 15  ✖️ FAILED: pod-network-latency chaos test failed 🗡️💀♻️ ✖️ FAILED: pod-network-latency chaos test failed 🗡️💀♻️ ✖️ FAILED: pod-network-latency chaos test failed 🗡️💀♻️ ✖️ FAILED: pod-network-latency chaos test failed 🗡️💀♻️ ✔️ PASSED: pod_network_latency chaos test passed 🗡️💀♻️ error: resource mapping not found for name: "burn-cpu" namespace: "default" from "/src/cnf-testsuite/cnfs/coredns/chaos_cpu_hog.yml": no matches for kind "StressChaos" in version "chaos-mesh.org/v1alpha1" ensure CRDs are installed first ✔️ PASSED: Application pod is healthy after high CPU consumption 📦💻🐷📈 error: resource mapping not found for name: "burn-cpu" namespace: "default" from "/src/cnf-testsuite/cnfs/coredns/chaos_cpu_hog.yml": no matches for kind "StressChaos" in version "chaos-mesh.org/v1alpha1" ensure CRDs are installed first error: resource mapping not found for name: "container-kill" namespace: "default" from "/src/cnf-testsuite/cnfs/coredns/chaos_container_kill.yml": no matches for kind "PodChaos" in version "chaos-mesh.org/v1alpha1" ensure CRDs are installed first ✔️ PASSED: Replicas available match desired count after container kill test 🗡️💀♻️ error: resource mapping not found for name: "container-kill" namespace: "default" from "/src/cnf-testsuite/cnfs/coredns/chaos_container_kill.yml": no matches for kind "PodChaos" in version "chaos-mesh.org/v1alpha1" ensure CRDs are installed first ✔️ PASSED: disk_fill chaos test passed 🗡️💀♻️ ✖️ FAILED: pod-delete chaos test failed 🗡️💀♻️ ✔️ PASSED: pod_delete chaos test passed 🗡️💀♻️ pod-memory-hog chaos test failed 🗡️💀♻️ ✖️ FAILED: pod_memory_hog chaos test failed 🗡️💀♻️ ✔️ PASSED: pod_io_stress chaos test passed 🗡️💀♻️ Resilience final score: 10 of 10  Final workload score: 119 of 146 CNFManager::Points::Results.have been saved to results/cnf-testsuite-results-20221014-140903-470.yml 2022-10-14 14:34:30,737 - functest_kubernetes.cnf_conformance.conformance - WARNING - secrets_used skipped 2022-10-14 14:34:30,738 - functest_kubernetes.cnf_conformance.conformance - WARNING - immutable_configmap failed 2022-10-14 14:34:30,738 - functest_kubernetes.cnf_conformance.conformance - WARNING - reasonable_startup_time failed 2022-10-14 14:34:30,738 - functest_kubernetes.cnf_conformance.conformance - WARNING - pod-network-latency failed 2022-10-14 14:34:30,738 - functest_kubernetes.cnf_conformance.conformance - WARNING - pod_memory_hog failed 2022-10-14 14:34:30,745 - functest_kubernetes.cnf_conformance.conformance - INFO - +-------------------------------------------------------------+-----------------+ | NAME | STATUS | +-------------------------------------------------------------+-----------------+ | volume_hostpath_not_found | passed | | privileged | passed | | non_root_user | passed | | increase_capacity | passed | | decrease_capacity | passed | | ip_addresses | passed | | liveness | passed | | readiness | passed | | nodeport_not_used | passed | | hardcoded_ip_addresses_in_k8s_runtime_configuration | passed | | rollback | passed | | secrets_used | skipped | | immutable_configmap | failed | | rolling_update | passed | | rolling_downgrade | passed | | rolling_version_change | passed | | install_script_helm | passed | | helm_chart_valid | passed | | helm_chart_published | passed | | reasonable_image_size | passed | | reasonable_startup_time | failed | | single_process_type | passed | | pod_network_latency | passed | | chaos_cpu_hog | passed | | chaos_container_kill | passed | | disk_fill | passed | | pod-network-latency | failed | | pod_delete | passed | | pod_memory_hog | failed | | pod_io_stress | passed | +-------------------------------------------------------------+-----------------+ 2022-10-14 14:34:30,746 - xtesting.ci.run_tests - INFO - Test result: +-----------------------+------------------+------------------+----------------+ | TEST CASE | PROJECT | DURATION | RESULT | +-----------------------+------------------+------------------+----------------+ | cnf_testsuite | functest | 26:19 | PASS | +-----------------------+------------------+------------------+----------------+ 2022-10-14 14:34:30,879 - functest_kubernetes.cnf_conformance.conformance - INFO - cnf-testsuite uninstall_cri_tools cnf-config=cnf-testsuite.yml 2022-10-14 14:34:31,467 - functest_kubernetes.cnf_conformance.conformance - INFO - cnf-testsuite uninstall_falco cnf-config=cnf-testsuite.yml 2022-10-14 14:34:31,886 - functest_kubernetes.cnf_conformance.conformance - INFO - cnf-testsuite cnf_cleanup cnf-config=cnf-testsuite.yml Successfully cleaned up coredns