2020-09-29 12:51:52,914 - xtesting.ci.run_tests - INFO - Deployment description: +-------------------------+------------------------------------------------------------+ | ENV VAR | VALUE | +-------------------------+------------------------------------------------------------+ | CI_LOOP | daily | | DEBUG | true | | DEPLOY_SCENARIO | k8-nosdn-nofeature-noha | | INSTALLER_TYPE | unknown | | BUILD_TAG | 0I3XDBUQ27AT | | NODE_NAME | lf-virtual1-4 | | TEST_DB_URL | http://testresults.opnfv.org/test/api/v1/results | | TEST_DB_EXT_URL | http://testresults.opnfv.org/test/api/v1/results | | S3_ENDPOINT_URL | https://storage.googleapis.com | | S3_DST_URL | s3://artifacts.opnfv.org/functest- | | | kubernetes/0I3XDBUQ27AT/functest-kubernetes-opnfv- | | | functest-kubernetes-cnf-kali-cnf_conformance-run-4 | | HTTP_DST_URL | http://artifacts.opnfv.org/functest- | | | kubernetes/0I3XDBUQ27AT/functest-kubernetes-opnfv- | | | functest-kubernetes-cnf-kali-cnf_conformance-run-4 | +-------------------------+------------------------------------------------------------+ 2020-09-29 12:51:52,929 - xtesting.ci.run_tests - INFO - Loading test case 'cnf_conformance'... 2020-09-29 12:51:52,951 - xtesting.ci.run_tests - INFO - Running test case 'cnf_conformance'... 2020-09-29 12:51:58,670 - functest_kubernetes.cnf_conformance.conformance - INFO - cnf-conformance setup % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 658 100 658 0 0 2300 0 --:--:-- --:--:-- --:--:-- 2308 0 9281k 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 9281k 100 9281k 0 0 5722k 0 0:00:01 0:00:01 --:--:-- 10.7M --2020-09-29 12:51:55-- https://get.helm.sh/helm-v3.1.1-linux-amd64.tar.gz Resolving get.helm.sh... 152.195.19.97, 2606:2800:11f:1cb7:261b:1f9c:2074:3c Connecting to get.helm.sh|152.195.19.97|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 12270615 (12M) [application/x-tar] Saving to: '/var/lib/xtesting/results/cnf_conformance/tools/helm/helm-v3.1.1-linux-amd64.tar.gz' 0K .......... .......... .......... .......... .......... 0% 2.39M 5s 50K .......... .......... .......... .......... .......... 0% 3.77M 4s 100K .......... .......... .......... .......... .......... 1% 7.05M 3s 150K .......... .......... .......... .......... .......... 1% 66.8M 2s 200K .......... .......... .......... .......... .......... 2% 8.51M 2s 250K .......... .......... .......... .......... .......... 2% 62.9M 2s 300K .......... .......... .......... .......... .......... 2% 10.7M 2s 350K .......... .......... .......... .......... .......... 3% 66.1M 2s 400K .......... .......... .......... .......... .......... 3% 86.9M 1s 450K .......... .......... .......... .......... .......... 4% 88.8M 1s 500K .......... .......... .......... .......... .......... 4% 11.7M 1s 550K .......... .......... .......... .......... .......... 5% 51.3M 1s 600K .......... .......... .......... .......... .......... 5% 42.9M 1s 650K .......... .......... .......... .......... .......... 5% 18.1M 1s 700K .......... .......... .......... .......... .......... 6% 36.0M 1s 750K .......... .......... .......... .......... .......... 6% 44.8M 1s 800K .......... .......... .......... .......... .......... 7% 41.8M 1s 850K .......... .......... .......... .......... .......... 7% 183M 1s 900K .......... .......... .......... .......... .......... 7% 15.0M 1s 950K .......... .......... .......... .......... .......... 8% 46.5M 1s 1000K .......... .......... .......... .......... .......... 8% 34.3M 1s 1050K .......... .......... .......... .......... .......... 9% 34.6M 1s 1100K .......... .......... .......... .......... .......... 9% 45.0M 1s 1150K .......... .......... .......... .......... .......... 10% 53.6M 1s 1200K .......... .......... .......... .......... .......... 10% 86.3M 1s 1250K .......... .......... .......... .......... .......... 10% 81.6M 1s 1300K .......... .......... .......... .......... .......... 11% 87.7M 1s 1350K .......... .......... .......... .......... .......... 11% 60.0M 1s 1400K .......... .......... .......... .......... .......... 12% 132M 1s 1450K .......... .......... .......... .......... .......... 12% 112M 1s 1500K .......... .......... .......... .......... .......... 12% 126M 1s 1550K .......... .......... .......... .......... .......... 13% 74.3M 1s 1600K .......... .......... .......... .......... .......... 13% 219M 1s 1650K .......... .......... .......... .......... .......... 14% 234M 0s 1700K .......... .......... .......... .......... .......... 14% 212M 0s 1750K .......... .......... .......... .......... .......... 15% 175M 0s 1800K .......... .......... .......... .......... .......... 15% 269M 0s 1850K .......... .......... .......... .......... .......... 15% 272M 0s 1900K .......... .......... .......... .......... .......... 16% 269M 0s 1950K .......... .......... .......... .......... .......... 16% 34.5M 0s 2000K .......... .......... .......... .......... .......... 17% 57.2M 0s 2050K .......... .......... .......... .......... .......... 17% 68.1M 0s 2100K .......... .......... .......... .......... .......... 17% 65.7M 0s 2150K .......... .......... .......... .......... .......... 18% 59.2M 0s 2200K .......... .......... .......... .......... .......... 18% 78.4M 0s 2250K .......... .......... .......... .......... .......... 19% 72.2M 0s 2300K .......... .......... .......... .......... .......... 19% 66.6M 0s 2350K .......... .......... .......... .......... .......... 20% 72.6M 0s 2400K .......... .......... .......... .......... .......... 20% 89.7M 0s 2450K .......... .......... .......... .......... .......... 20% 80.8M 0s 2500K .......... .......... .......... .......... .......... 21% 81.8M 0s 2550K .......... .......... .......... .......... .......... 21% 77.8M 0s 2600K .......... .......... .......... .......... .......... 22% 108M 0s 2650K .......... .......... .......... .......... .......... 22% 128M 0s 2700K .......... .......... .......... .......... .......... 22% 109M 0s 2750K .......... .......... .......... .......... .......... 23% 101M 0s 2800K .......... .......... .......... .......... .......... 23% 126M 0s 2850K .......... .......... .......... .......... .......... 24% 106M 0s 2900K .......... .......... .......... .......... .......... 24% 107M 0s 2950K .......... .......... .......... .......... .......... 25% 95.0M 0s 3000K .......... .......... .......... .......... .......... 25% 2.25M 0s 3050K .......... .......... .......... .......... .......... 25% 198M 0s 3100K .......... .......... .......... .......... .......... 26% 264M 0s 3150K .......... .......... .......... .......... .......... 26% 195M 0s 3200K .......... .......... .......... .......... .......... 27% 247M 0s 3250K .......... .......... .......... .......... .......... 27% 269M 0s 3300K .......... .......... .......... .......... .......... 27% 268M 0s 3350K .......... .......... .......... .......... .......... 28% 244M 0s 3400K .......... .......... .......... .......... .......... 28% 268M 0s 3450K .......... .......... .......... .......... .......... 29% 251M 0s 3500K .......... .......... .......... .......... .......... 29% 267M 0s 3550K .......... .......... .......... .......... .......... 30% 2.08M 0s 3600K .......... .......... .......... .......... .......... 30% 82.4M 0s 3650K .......... .......... .......... .......... .......... 30% 134M 0s 3700K .......... .......... .......... .......... .......... 31% 115M 0s 3750K .......... .......... .......... .......... .......... 31% 137M 0s 3800K .......... .......... .......... .......... .......... 32% 104M 0s 3850K .......... .......... .......... .......... .......... 32% 159M 0s 3900K .......... .......... .......... .......... .......... 32% 135M 0s 3950K .......... .......... .......... .......... .......... 33% 137M 0s 4000K .......... .......... .......... .......... .......... 33% 186M 0s 4050K .......... .......... .......... .......... .......... 34% 181M 0s 4100K .......... .......... .......... .......... .......... 34% 168M 0s 4150K .......... .......... .......... .......... .......... 35% 161M 0s 4200K .......... .......... .......... .......... .......... 35% 118M 0s 4250K .......... .......... .......... .......... .......... 35% 90.6M 0s 4300K .......... .......... .......... .......... .......... 36% 207M 0s 4350K .......... .......... .......... .......... .......... 36% 84.6M 0s 4400K .......... .......... .......... .......... .......... 37% 135M 0s 4450K .......... .......... .......... .......... .......... 37% 136M 0s 4500K .......... .......... .......... .......... .......... 37% 109M 0s 4550K .......... .......... .......... .......... .......... 38% 70.6M 0s 4600K .......... .......... .......... .......... .......... 38% 81.9M 0s 4650K .......... .......... .......... .......... .......... 39% 215M 0s 4700K .......... .......... .......... .......... .......... 39% 235M 0s 4750K .......... .......... .......... .......... .......... 40% 175M 0s 4800K .......... .......... .......... .......... .......... 40% 237M 0s 4850K .......... .......... .......... .......... .......... 40% 212M 0s 4900K .......... .......... .......... .......... .......... 41% 201M 0s 4950K .......... .......... .......... .......... .......... 41% 200M 0s 5000K .......... .......... .......... .......... .......... 42% 278M 0s 5050K .......... .......... .......... .......... .......... 42% 245M 0s 5100K .......... .......... .......... .......... .......... 42% 281M 0s 5150K .......... .......... .......... .......... .......... 43% 233M 0s 5200K .......... .......... .......... .......... .......... 43% 196M 0s 5250K .......... .......... .......... .......... .......... 44% 272M 0s 5300K .......... .......... .......... .......... .......... 44% 241M 0s 5350K .......... .......... .......... .......... .......... 45% 202M 0s 5400K .......... .......... .......... .......... .......... 45% 260M 0s 5450K .......... .......... .......... .......... .......... 45% 234M 0s 5500K .......... .......... .......... .......... .......... 46% 283M 0s 5550K .......... .......... .......... .......... .......... 46% 222M 0s 5600K .......... .......... .......... .......... .......... 47% 270M 0s 5650K .......... .......... .......... .......... .......... 47% 246M 0s 5700K .......... .......... .......... .......... .......... 47% 200M 0s 5750K .......... .......... .......... .......... .......... 48% 237M 0s 5800K .......... .......... .......... .......... .......... 48% 270M 0s 5850K .......... .......... .......... .......... .......... 49% 242M 0s 5900K .......... .......... .......... .......... .......... 49% 270M 0s 5950K .......... .......... .......... .......... .......... 50% 208M 0s 6000K .......... .......... .......... .......... .......... 50% 271M 0s 6050K .......... .......... .......... .......... .......... 50% 255M 0s 6100K .......... .......... .......... .......... .......... 51% 276M 0s 6150K .......... .......... .......... .......... .......... 51% 224M 0s 6200K .......... .......... .......... .......... .......... 52% 253M 0s 6250K .......... .......... .......... .......... .......... 52% 7.02M 0s 6300K .......... .......... .......... .......... .......... 52% 68.6M 0s 6350K .......... .......... .......... .......... .......... 53% 63.6M 0s 6400K .......... .......... .......... .......... .......... 53% 96.3M 0s 6450K .......... .......... .......... .......... .......... 54% 87.9M 0s 6500K .......... .......... .......... .......... .......... 54% 73.0M 0s 6550K .......... .......... .......... .......... .......... 55% 84.3M 0s 6600K .......... .......... .......... .......... .......... 55% 98.5M 0s 6650K .......... .......... .......... .......... .......... 55% 89.7M 0s 6700K .......... .......... .......... .......... .......... 56% 89.5M 0s 6750K .......... .......... .......... .......... .......... 56% 79.6M 0s 6800K .......... .......... .......... .......... .......... 57% 87.6M 0s 6850K .......... .......... .......... .......... .......... 57% 45.7M 0s 6900K .......... .......... .......... .......... .......... 57% 98.7M 0s 6950K .......... .......... .......... .......... .......... 58% 84.3M 0s 7000K .......... .......... .......... .......... .......... 58% 89.7M 0s 7050K .......... .......... .......... .......... .......... 59% 87.7M 0s 7100K .......... .......... .......... .......... .......... 59% 99.0M 0s 7150K .......... .......... .......... .......... .......... 60% 83.6M 0s 7200K .......... .......... .......... .......... .......... 60% 68.2M 0s 7250K .......... .......... .......... .......... .......... 60% 90.7M 0s 7300K .......... .......... .......... .......... .......... 61% 154M 0s 7350K .......... .......... .......... .......... .......... 61% 238M 0s 7400K .......... .......... .......... .......... .......... 62% 258M 0s 7450K .......... .......... .......... .......... .......... 62% 233M 0s 7500K .......... .......... .......... .......... .......... 63% 241M 0s 7550K .......... .......... .......... .......... .......... 63% 211M 0s 7600K .......... .......... .......... .......... .......... 63% 17.6M 0s 7650K .......... .......... .......... .......... .......... 64% 90.5M 0s 7700K .......... .......... .......... .......... .......... 64% 104M 0s 7750K .......... .......... .......... .......... .......... 65% 84.2M 0s 7800K .......... .......... .......... .......... .......... 65% 101M 0s 7850K .......... .......... .......... .......... .......... 65% 104M 0s 7900K .......... .......... .......... .......... .......... 66% 68.8M 0s 7950K .......... .......... .......... .......... .......... 66% 91.4M 0s 8000K .......... .......... .......... .......... .......... 67% 102M 0s 8050K .......... .......... .......... .......... .......... 67% 100M 0s 8100K .......... .......... .......... .......... .......... 68% 116M 0s 8150K .......... .......... .......... .......... .......... 68% 100M 0s 8200K .......... .......... .......... .......... .......... 68% 4.61M 0s 8250K .......... .......... .......... .......... .......... 69% 157M 0s 8300K .......... .......... .......... .......... .......... 69% 241M 0s 8350K .......... .......... .......... .......... .......... 70% 182M 0s 8400K .......... .......... .......... .......... .......... 70% 224M 0s 8450K .......... .......... .......... .......... .......... 70% 200M 0s 8500K .......... .......... .......... .......... .......... 71% 196M 0s 8550K .......... .......... .......... .......... .......... 71% 204M 0s 8600K .......... .......... .......... .......... .......... 72% 229M 0s 8650K .......... .......... .......... .......... .......... 72% 226M 0s 8700K .......... .......... .......... .......... .......... 73% 213M 0s 8750K .......... .......... .......... .......... .......... 73% 123M 0s 8800K .......... .......... .......... .......... .......... 73% 226M 0s 8850K .......... .......... .......... .......... .......... 74% 218M 0s 8900K .......... .......... .......... .......... .......... 74% 227M 0s 8950K .......... .......... .......... .......... .......... 75% 205M 0s 9000K .......... .......... .......... .......... .......... 75% 201M 0s 9050K .......... .......... .......... .......... .......... 75% 226M 0s 9100K .......... .......... .......... .......... .......... 76% 217M 0s 9150K .......... .......... .......... .......... .......... 76% 174M 0s 9200K .......... .......... .......... .......... .......... 77% 208M 0s 9250K .......... .......... .......... .......... .......... 77% 203M 0s 9300K .......... .......... .......... .......... .......... 78% 231M 0s 9350K .......... .......... .......... .......... .......... 78% 199M 0s 9400K .......... .......... .......... .......... .......... 78% 216M 0s 9450K .......... .......... .......... .......... .......... 79% 220M 0s 9500K .......... .......... .......... .......... .......... 79% 225M 0s 9550K .......... .......... .......... .......... .......... 80% 119M 0s 9600K .......... .......... .......... .......... .......... 80% 177M 0s 9650K .......... .......... .......... .......... .......... 80% 219M 0s 9700K .......... .......... .......... .......... .......... 81% 215M 0s 9750K .......... .......... .......... .......... .......... 81% 178M 0s 9800K .......... .......... .......... .......... .......... 82% 181M 0s 9850K .......... .......... .......... .......... .......... 82% 209M 0s 9900K .......... .......... .......... .......... .......... 83% 226M 0s 9950K .......... .......... .......... .......... .......... 83% 181M 0s 10000K .......... .......... .......... .......... .......... 83% 216M 0s 10050K .......... .......... .......... .......... .......... 84% 178M 0s 10100K .......... .......... .......... .......... .......... 84% 218M 0s 10150K .......... .......... .......... .......... .......... 85% 204M 0s 10200K .......... .......... .......... .......... .......... 85% 208M 0s 10250K .......... .......... .......... .......... .......... 85% 226M 0s 10300K .......... .......... .......... .......... .......... 86% 179M 0s 10350K .......... .......... .......... .......... .......... 86% 146M 0s 10400K .......... .......... .......... .......... .......... 87% 199M 0s 10450K .......... .......... .......... .......... .......... 87% 184M 0s 10500K .......... .......... .......... .......... .......... 88% 202M 0s 10550K .......... .......... .......... .......... .......... 88% 185M 0s 10600K .......... .......... .......... .......... .......... 88% 219M 0s 10650K .......... .......... .......... .......... .......... 89% 228M 0s 10700K .......... .......... .......... .......... .......... 89% 186M 0s 10750K .......... .......... .......... .......... .......... 90% 246M 0s 10800K .......... .......... .......... .......... .......... 90% 268M 0s 10850K .......... .......... .......... .......... .......... 90% 44.1M 0s 10900K .......... .......... .......... .......... .......... 91% 43.0M 0s 10950K .......... .......... .......... .......... .......... 91% 79.3M 0s 11000K .......... .......... .......... .......... .......... 92% 85.8M 0s 11050K .......... .......... .......... .......... .......... 92% 82.5M 0s 11100K .......... .......... .......... .......... .......... 93% 84.4M 0s 11150K .......... .......... .......... .......... .......... 93% 52.4M 0s 11200K .......... .......... .......... .......... .......... 93% 55.6M 0s 11250K .......... .......... .......... .......... .......... 94% 4.39M 0s 11300K .......... .......... .......... .......... .......... 94% 162M 0s 11350K .......... .......... .......... .......... .......... 95% 268M 0s 11400K .......... .......... .......... .......... .......... 95% 262M 0s 11450K .......... .......... .......... .......... .......... 95% 287M 0s 11500K .......... .......... .......... .......... .......... 96% 287M 0s 11550K .......... .......... .......... .......... .......... 96% 260M 0s 11600K .......... .......... .......... .......... .......... 97% 235M 0s 11650K .......... .......... .......... .......... .......... 97% 283M 0s 11700K .......... .......... .......... .......... .......... 98% 300M 0s 11750K .......... .......... .......... .......... .......... 98% 252M 0s 11800K .......... .......... .......... .......... .......... 98% 302M 0s 11850K .......... .......... .......... .......... .......... 99% 264M 0s 11900K .......... .......... .......... .......... .......... 99% 298M 0s 11950K .......... .......... .......... ... 100% 298M=0.2s 2020-09-29 12:51:56 (50.9 MB/s) - '/var/lib/xtesting/results/cnf_conformance/tools/helm/helm-v3.1.1-linux-amd64.tar.gz' saved [12270615/12270615] Global helm found. Version: v3.3.1 Local helm found. Version:  Global wget found. Version: 1.20.3 No Local wget version found Global curl found. Version: 7.67.0 No Local curl version found Global kubectl found. Version: 1.19+ No Local kubectl version found All prerequisites found. create_results_yml file exists?:false Successfully created directories for cnf-conformance Setup complete 2020-09-29 12:52:36,797 - functest_kubernetes.cnf_conformance.conformance - INFO - cnf-conformance cnf_setup cnf-config=cnf-conformance.yml Successfully created directories for cnf-conformance Successfully setup coredns 2020-09-29 12:54:54,807 - functest_kubernetes.cnf_conformance.conformance - INFO - cnf-conformance all install.go:158: [debug] Original chart version: "" install.go:175: [debug] CHART PATH: /root/.cache/helm/repository/coredns-1.13.3.tgz Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply create_results_yml file exists?:false ✔️ PASSED: No privileged containers Security final score: 5 of 5 ✔️ PASSED: Replicas increased to 3 📦📈 ✔️ PASSED: Replicas decreased to 1 📦📉 Scalability final score: 20 of 20 ✖️ FAILURE: IP addresses found ✔️ PASSED: Helm liveness probe found ✔️ PASSED: Helm readiness probe found ✔️ PASSED: CNF coredns-coredns Rolling Update Passed ✔️ PASSED: NodePort is not used ✔️ PASSED: No hard-coded IP addresses found in the runtime K8s configuration Configuration lifecycle final score: 24 of 25 PASSED (by default): No install script provided Successfully created directories for cnf-conformance ✔️ PASSED: Helm Chart cnfs/coredns/helm_chart/coredns Lint Passed ✔️ PASSED: Published Helm Chart Found ✔️ PASSED: Helm deploy successful Installability final score: 20 of 20 Hostname lookup for hub.docker.com failed: System error /home/pair/workspace/taylor/qa/jun8/lib/halite/src/halite/client.cr:155:7 in 'perform' /home/pair/workspace/taylor/qa/jun8/lib/halite/src/halite/client.cr:115:9 in 'request' /home/pair/workspace/taylor/qa/jun8/lib/halite/src/halite/chainable.cr:485:7 in '->' /home/pair/workspace/taylor/qa/jun8/src/tasks/utils/utils.cr:255:3 in 'task_runner' /home/pair/workspace/taylor/qa/jun8/src/tasks/microservice.cr:93:3 in '->' /home/pair/workspace/taylor/qa/jun8/lib/sam/src/sam/task.cr:39:29 in 'call' /home/pair/workspace/taylor/qa/jun8/lib/sam/src/sam/task.cr:39:29 in 'call' /home/pair/workspace/taylor/qa/jun8/lib/sam/src/sam.cr:43:5 in '__crystal_main' /usr/local/depot/crystal-0.33.0-1/share/crystal/src/crystal/main.cr:106:5 in 'main' ??? ??? ✔️ PASSED: CNF had a reasonable startup time 🚀 Microservice final score: 5 of 10 Final score: 74 of 80 Results have been saved to cnf-conformance-results-20200929-125454-801.yml 2020-09-29 12:54:54,823 - functest_kubernetes.cnf_conformance.conformance - INFO - +-------------------------------------------------------------+----------------+ | NAME | STATUS | +-------------------------------------------------------------+----------------+ | privileged | passed | | increase_capacity | passed | | decrease_capacity | passed | | ip_addresses | failed | | liveness | passed | | readiness | passed | | rolling_update | passed | | nodeport_not_used | passed | | hardcoded_ip_addresses_in_k8s_runtime_configuration | passed | | install_script_helm | passed | | helm_chart_valid | passed | | helm_chart_published | passed | | helm_deploy | passed | | reasonable_startup_time | passed | +-------------------------------------------------------------+----------------+ 2020-09-29 12:54:54,827 - xtesting.ci.run_tests - INFO - Test result: +-------------------------+------------------+------------------+----------------+ | TEST CASE | PROJECT | DURATION | RESULT | +-------------------------+------------------+------------------+----------------+ | cnf_conformance | functest | 03:01 | PASS | +-------------------------+------------------+------------------+----------------+ 2020-09-29 12:55:04,253 - functest_kubernetes.cnf_conformance.conformance - INFO - cnf-conformance cnf_cleanup cnf-config=cnf-conformance.yml Successfully cleaned up coredns