2021-06-14 18:53:01,869 - xtesting.ci.run_tests - INFO - Deployment description: +-------------------------+----------------------------------------------------------+ | ENV VAR | VALUE | +-------------------------+----------------------------------------------------------+ | CI_LOOP | daily | | DEBUG | false | | DEPLOY_SCENARIO | k8-nosdn-nofeature-noha | | INSTALLER_TYPE | unknown | | BUILD_TAG | 0H3Y7UABWUEF | | NODE_NAME | lf-virtual1-5 | | TEST_DB_URL | http://testresults.opnfv.org/test/api/v1/results | | TEST_DB_EXT_URL | http://testresults.opnfv.org/test/api/v1/results | | S3_ENDPOINT_URL | https://storage.googleapis.com | | S3_DST_URL | s3://artifacts.opnfv.org/functest- | | | kubernetes/0H3Y7UABWUEF/functest-kubernetes-pi- | | | ollivier-functest-kubernetes-cnf-arm-leguer- | | | cnf_testsuite-run-2 | | HTTP_DST_URL | http://artifacts.opnfv.org/functest- | | | kubernetes/0H3Y7UABWUEF/functest-kubernetes-pi- | | | ollivier-functest-kubernetes-cnf-arm-leguer- | | | cnf_testsuite-run-2 | +-------------------------+----------------------------------------------------------+ 2021-06-14 18:53:01,884 - xtesting.ci.run_tests - INFO - Loading test case 'cnf_testsuite'... 2021-06-14 18:53:01,951 - xtesting.ci.run_tests - INFO - Running test case 'cnf_testsuite'... 2021-06-14 18:53:03,810 - functest_kubernetes.cnf_conformance.conformance - INFO - cnf-testsuite setup Successfully created directories for cnf-testsuite Global helm found. Version: v3.3.1 Local helm found. Version: v3.3.1 Global kubectl found. Version: 1.20 No Local kubectl version found Global git found. Version: 2.30.2 No Local git version found All prerequisites found. Setup complete 2021-06-14 18:53:25,749 - functest_kubernetes.cnf_conformance.conformance - INFO - cnf-testsuite cnf_setup cnf-config=cnf-testsuite.yml Successfully created directories for cnf-testsuite Successfully setup coredns 2021-06-14 18:59:39,818 - functest_kubernetes.cnf_conformance.conformance - INFO - cnf-testsuite workload ✔️ PASSED: hostPath volumes not found 🖥️ 💾 Statelessness final score: 5 of 10  ✔️ PASSED: No privileged containers 🔓🔑 Security final score: 5 of 5  ✔️ PASSED: Replicas increased to 3 📦📈 ✔️ PASSED: Replicas decreased to 1 📦📉 Scalability final score: 20 of 35  ✔️ PASSED: No IP addresses found ✔️ PASSED: Helm liveness probe found ⎈🧫 ✔️ PASSED: Helm readiness probe found ⎈🧫 ✔️ PASSED: NodePort is not used ✔️ PASSED: No hard-coded IP addresses found in the runtime K8s configuration ✔️ PASSED: CNF Rollback Passed No Secret Volumes or Container secretKeyRefs found for resource: {kind: "Deployment", name: "coredns-coredns"} ⏭ SKIPPED: Secrets not used 🧫 To address this issue please see the USAGE.md documentation  ✖️ FAILED: Found mutable configmap(s) ⚖️ ✔️ PASSED: CNF for Rolling Update Passed ✔️ PASSED: CNF for Rolling Downgrade Passed ✔️ PASSED: CNF for Rolling Version Change Passed Configuration Lifecycle final score: 40 of 46  ✔️ PASSED (by default): No install script provided Successfully created directories for cnf-testsuite ✔️ PASSED: Helm Chart exported_chart Lint Passed ⎈📝☑️ ✖️ FAILED: Published Helm Chart Not Found ⎈📦🌐 SKIPPED: Helm Deploy Installability final score: 9 of 20  ✔️ PASSED: Image size is good 🐜 ⚖️👀 ✔️ PASSED: CNF had a reasonable startup time 🚀 Microservice final score: 10 of 10  ✔️ PASSED: pod_network_latency chaos test passed 🗡️💀♻️ ✔️ PASSED: Application pod is healthy after high CPU consumption 📦💻🐷📈 ✔️ PASSED: Replicas available match desired count after container kill test 🗡️💀♻️ ✔️ PASSED: disk_fill chaos test passed 🗡️💀♻️ Resilience final score: 10 of 10  Final workload score: 109 of 136 CNFManager::Points::Results.have been saved to results/cnf-testsuite-results-20210614-185325-759.yml 2021-06-14 18:59:39,856 - functest_kubernetes.cnf_conformance.conformance - INFO - +-------------------------------------------------------------+-----------------+ | NAME | STATUS | +-------------------------------------------------------------+-----------------+ | volume_hostpath_not_found | passed | | privileged | passed | | increase_capacity | passed | | decrease_capacity | passed | | ip_addresses | passed | | liveness | passed | | readiness | passed | | nodeport_not_used | passed | | hardcoded_ip_addresses_in_k8s_runtime_configuration | passed | | rollback | passed | | secrets_used | skipped | | immutable_configmap | failed | | rolling_update | passed | | rolling_downgrade | passed | | rolling_version_change | passed | | install_script_helm | passed | | helm_chart_valid | passed | | helm_chart_published | failed | | reasonable_image_size | passed | | reasonable_startup_time | passed | | pod_network_latency | passed | | chaos_cpu_hog | passed | | chaos_container_kill | passed | | disk_fill | passed | +-------------------------------------------------------------+-----------------+ 2021-06-14 18:59:39,857 - xtesting.ci.run_tests - INFO - Test result: +-----------------------+------------------+------------------+----------------+ | TEST CASE | PROJECT | DURATION | RESULT | +-----------------------+------------------+------------------+----------------+ | cnf_testsuite | functest | 06:37 | PASS | +-----------------------+------------------+------------------+----------------+ 2021-06-14 18:59:40,418 - functest_kubernetes.cnf_conformance.conformance - INFO - cnf-testsuite cnf_cleanup cnf-config=cnf-testsuite.yml Successfully cleaned up coredns