2020-10-27 12:48:51,917 - xtesting.ci.run_tests - INFO - Deployment description: +-------------------------+------------------------------------------------------------+ | ENV VAR | VALUE | +-------------------------+------------------------------------------------------------+ | CI_LOOP | daily | | DEBUG | true | | DEPLOY_SCENARIO | k8-nosdn-nofeature-noha | | INSTALLER_TYPE | unknown | | BUILD_TAG | 1KGG4YISVV5T | | NODE_NAME | lf-virtual1-4 | | TEST_DB_URL | http://testresults.opnfv.org/test/api/v1/results | | TEST_DB_EXT_URL | http://testresults.opnfv.org/test/api/v1/results | | S3_ENDPOINT_URL | https://storage.googleapis.com | | S3_DST_URL | s3://artifacts.opnfv.org/functest- | | | kubernetes/1KGG4YISVV5T/functest-kubernetes-opnfv- | | | functest-kubernetes-cnf-kali-cnf_conformance-run-8 | | HTTP_DST_URL | http://artifacts.opnfv.org/functest- | | | kubernetes/1KGG4YISVV5T/functest-kubernetes-opnfv- | | | functest-kubernetes-cnf-kali-cnf_conformance-run-8 | +-------------------------+------------------------------------------------------------+ 2020-10-27 12:48:51,931 - xtesting.ci.run_tests - INFO - Loading test case 'cnf_conformance'... 2020-10-27 12:48:51,954 - xtesting.ci.run_tests - INFO - Running test case 'cnf_conformance'... 2020-10-27 12:49:03,095 - functest_kubernetes.cnf_conformance.conformance - INFO - cnf-conformance setup % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 658 100 658 0 0 2464 0 --:--:-- --:--:-- --:--:-- 2464 0 9281k 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 32 9281k 32 3052k 0 0 1820k 0 0:00:05 0:00:01 0:00:04 3457k 96 9281k 96 8972k 0 0 3398k 0 0:00:02 0:00:02 --:--:-- 4860k 100 9281k 100 9281k 0 0 3504k 0 0:00:02 0:00:02 --:--:-- 5005k --2020-10-27 12:48:55-- https://get.helm.sh/helm-v3.1.1-linux-amd64.tar.gz Resolving get.helm.sh... 152.195.19.97, 2606:2800:11f:1cb7:261b:1f9c:2074:3c Connecting to get.helm.sh|152.195.19.97|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 12270615 (12M) [application/x-tar] Saving to: '/var/lib/xtesting/results/cnf_conformance/tools/helm/helm-v3.1.1-linux-amd64.tar.gz' 0K .......... .......... .......... .......... .......... 0% 2.42M 5s 50K .......... .......... .......... .......... .......... 0% 2.63M 5s 100K .......... .......... .......... .......... .......... 1% 54.9M 3s 150K .......... .......... .......... .......... .......... 1% 59.5M 2s 200K .......... .......... .......... .......... .......... 2% 3.14M 3s 250K .......... .......... .......... .......... .......... 2% 28.1M 2s 300K .......... .......... .......... .......... .......... 2% 98.3M 2s 350K .......... .......... .......... .......... .......... 3% 51.8M 2s 400K .......... .......... .......... .......... .......... 3% 42.1M 2s 450K .......... .......... .......... .......... .......... 4% 76.7M 1s 500K .......... .......... .......... .......... .......... 4% 3.15M 2s 550K .......... .......... .......... .......... .......... 5% 42.5M 1s 600K .......... .......... .......... .......... .......... 5% 63.9M 1s 650K .......... .......... .......... .......... .......... 5% 60.7M 1s 700K .......... .......... .......... .......... .......... 6% 55.8M 1s 750K .......... .......... .......... .......... .......... 6% 70.1M 1s 800K .......... .......... .......... .......... .......... 7% 90.2M 1s 850K .......... .......... .......... .......... .......... 7% 80.1M 1s 900K .......... .......... .......... .......... .......... 7% 76.4M 1s 950K .......... .......... .......... .......... .......... 8% 77.2M 1s 1000K .......... .......... .......... .......... .......... 8% 111M 1s 1050K .......... .......... .......... .......... .......... 9% 100M 1s 1100K .......... .......... .......... .......... .......... 9% 4.07M 1s 1150K .......... .......... .......... .......... .......... 10% 47.3M 1s 1200K .......... .......... .......... .......... .......... 10% 59.1M 1s 1250K .......... .......... .......... .......... .......... 10% 60.2M 1s 1300K .......... .......... .......... .......... .......... 11% 76.5M 1s 1350K .......... .......... .......... .......... .......... 11% 56.8M 1s 1400K .......... .......... .......... .......... .......... 12% 79.4M 1s 1450K .......... .......... .......... .......... .......... 12% 57.4M 1s 1500K .......... .......... .......... .......... .......... 12% 68.9M 1s 1550K .......... .......... .......... .......... .......... 13% 48.7M 1s 1600K .......... .......... .......... .......... .......... 13% 62.8M 1s 1650K .......... .......... .......... .......... .......... 14% 70.3M 1s 1700K .......... .......... .......... .......... .......... 14% 61.1M 1s 1750K .......... .......... .......... .......... .......... 15% 81.2M 1s 1800K .......... .......... .......... .......... .......... 15% 98.8M 1s 1850K .......... .......... .......... .......... .......... 15% 101M 1s 1900K .......... .......... .......... .......... .......... 16% 59.1M 1s 1950K .......... .......... .......... .......... .......... 16% 59.6M 1s 2000K .......... .......... .......... .......... .......... 17% 1.25M 1s 2050K .......... .......... .......... .......... .......... 17% 85.2M 1s 2100K .......... .......... .......... .......... .......... 17% 108M 1s 2150K .......... .......... .......... .......... .......... 18% 103M 1s 2200K .......... .......... .......... .......... .......... 18% 117M 1s 2250K .......... .......... .......... .......... .......... 19% 119M 1s 2300K .......... .......... .......... .......... .......... 19% 120M 1s 2350K .......... .......... .......... .......... .......... 20% 67.2M 1s 2400K .......... .......... .......... .......... .......... 20% 62.3M 1s 2450K .......... .......... .......... .......... .......... 20% 49.9M 1s 2500K .......... .......... .......... .......... .......... 21% 72.3M 1s 2550K .......... .......... .......... .......... .......... 21% 58.6M 1s 2600K .......... .......... .......... .......... .......... 22% 61.1M 1s 2650K .......... .......... .......... .......... .......... 22% 160M 1s 2700K .......... .......... .......... .......... .......... 22% 198M 1s 2750K .......... .......... .......... .......... .......... 23% 171M 1s 2800K .......... .......... .......... .......... .......... 23% 221M 1s 2850K .......... .......... .......... .......... .......... 24% 216M 0s 2900K .......... .......... .......... .......... .......... 24% 218M 0s 2950K .......... .......... .......... .......... .......... 25% 200M 0s 3000K .......... .......... .......... .......... .......... 25% 239M 0s 3050K .......... .......... .......... .......... .......... 25% 211M 0s 3100K .......... .......... .......... .......... .......... 26% 199M 0s 3150K .......... .......... .......... .......... .......... 26% 194M 0s 3200K .......... .......... .......... .......... .......... 27% 229M 0s 3250K .......... .......... .......... .......... .......... 27% 242M 0s 3300K .......... .......... .......... .......... .......... 27% 234M 0s 3350K .......... .......... .......... .......... .......... 28% 195M 0s 3400K .......... .......... .......... .......... .......... 28% 238M 0s 3450K .......... .......... .......... .......... .......... 29% 209M 0s 3500K .......... .......... .......... .......... .......... 29% 166M 0s 3550K .......... .......... .......... .......... .......... 30% 149M 0s 3600K .......... .......... .......... .......... .......... 30% 216M 0s 3650K .......... .......... .......... .......... .......... 30% 211M 0s 3700K .......... .......... .......... .......... .......... 31% 209M 0s 3750K .......... .......... .......... .......... .......... 31% 218M 0s 3800K .......... .......... .......... .......... .......... 32% 206M 0s 3850K .......... .......... .......... .......... .......... 32% 203M 0s 3900K .......... .......... .......... .......... .......... 32% 149M 0s 3950K .......... .......... .......... .......... .......... 33% 127M 0s 4000K .......... .......... .......... .......... .......... 33% 202M 0s 4050K .......... .......... .......... .......... .......... 34% 8.71M 0s 4100K .......... .......... .......... .......... .......... 34% 172M 0s 4150K .......... .......... .......... .......... .......... 35% 123M 0s 4200K .......... .......... .......... .......... .......... 35% 113M 0s 4250K .......... .......... .......... .......... .......... 35% 69.1M 0s 4300K .......... .......... .......... .......... .......... 36% 100M 0s 4350K .......... .......... .......... .......... .......... 36% 82.0M 0s 4400K .......... .......... .......... .......... .......... 37% 137M 0s 4450K .......... .......... .......... .......... .......... 37% 21.0M 0s 4500K .......... .......... .......... .......... .......... 37% 97.7M 0s 4550K .......... .......... .......... .......... .......... 38% 92.5M 0s 4600K .......... .......... .......... .......... .......... 38% 104M 0s 4650K .......... .......... .......... .......... .......... 39% 86.8M 0s 4700K .......... .......... .......... .......... .......... 39% 102M 0s 4750K .......... .......... .......... .......... .......... 40% 75.8M 0s 4800K .......... .......... .......... .......... .......... 40% 97.5M 0s 4850K .......... .......... .......... .......... .......... 40% 105M 0s 4900K .......... .......... .......... .......... .......... 41% 109M 0s 4950K .......... .......... .......... .......... .......... 41% 115M 0s 5000K .......... .......... .......... .......... .......... 42% 113M 0s 5050K .......... .......... .......... .......... .......... 42% 125M 0s 5100K .......... .......... .......... .......... .......... 42% 119M 0s 5150K .......... .......... .......... .......... .......... 43% 105M 0s 5200K .......... .......... .......... .......... .......... 43% 131M 0s 5250K .......... .......... .......... .......... .......... 44% 111M 0s 5300K .......... .......... .......... .......... .......... 44% 129M 0s 5350K .......... .......... .......... .......... .......... 45% 119M 0s 5400K .......... .......... .......... .......... .......... 45% 133M 0s 5450K .......... .......... .......... .......... .......... 45% 137M 0s 5500K .......... .......... .......... .......... .......... 46% 115M 0s 5550K .......... .......... .......... .......... .......... 46% 118M 0s 5600K .......... .......... .......... .......... .......... 47% 8.86M 0s 5650K .......... .......... .......... .......... .......... 47% 44.0M 0s 5700K .......... .......... .......... .......... .......... 47% 48.2M 0s 5750K .......... .......... .......... .......... .......... 48% 45.5M 0s 5800K .......... .......... .......... .......... .......... 48% 65.1M 0s 5850K .......... .......... .......... .......... .......... 49% 89.8M 0s 5900K .......... .......... .......... .......... .......... 49% 78.0M 0s 5950K .......... .......... .......... .......... .......... 50% 68.7M 0s 6000K .......... .......... .......... .......... .......... 50% 69.8M 0s 6050K .......... .......... .......... .......... .......... 50% 132M 0s 6100K .......... .......... .......... .......... .......... 51% 150M 0s 6150K .......... .......... .......... .......... .......... 51% 144M 0s 6200K .......... .......... .......... .......... .......... 52% 159M 0s 6250K .......... .......... .......... .......... .......... 52% 146M 0s 6300K .......... .......... .......... .......... .......... 52% 161M 0s 6350K .......... .......... .......... .......... .......... 53% 91.5M 0s 6400K .......... .......... .......... .......... .......... 53% 113M 0s 6450K .......... .......... .......... .......... .......... 54% 113M 0s 6500K .......... .......... .......... .......... .......... 54% 89.0M 0s 6550K .......... .......... .......... .......... .......... 55% 81.2M 0s 6600K .......... .......... .......... .......... .......... 55% 71.8M 0s 6650K .......... .......... .......... .......... .......... 55% 89.7M 0s 6700K .......... .......... .......... .......... .......... 56% 57.6M 0s 6750K .......... .......... .......... .......... .......... 56% 69.0M 0s 6800K .......... .......... .......... .......... .......... 57% 59.8M 0s 6850K .......... .......... .......... .......... .......... 57% 99.0M 0s 6900K .......... .......... .......... .......... .......... 57% 69.6M 0s 6950K .......... .......... .......... .......... .......... 58% 72.2M 0s 7000K .......... .......... .......... .......... .......... 58% 83.2M 0s 7050K .......... .......... .......... .......... .......... 59% 81.6M 0s 7100K .......... .......... .......... .......... .......... 59% 9.25M 0s 7150K .......... .......... .......... .......... .......... 60% 56.1M 0s 7200K .......... .......... .......... .......... .......... 60% 82.9M 0s 7250K .......... .......... .......... .......... .......... 60% 84.7M 0s 7300K .......... .......... .......... .......... .......... 61% 76.8M 0s 7350K .......... .......... .......... .......... .......... 61% 84.6M 0s 7400K .......... .......... .......... .......... .......... 62% 72.2M 0s 7450K .......... .......... .......... .......... .......... 62% 92.4M 0s 7500K .......... .......... .......... .......... .......... 63% 89.8M 0s 7550K .......... .......... .......... .......... .......... 63% 70.1M 0s 7600K .......... .......... .......... .......... .......... 63% 91.2M 0s 7650K .......... .......... .......... .......... .......... 64% 80.9M 0s 7700K .......... .......... .......... .......... .......... 64% 82.3M 0s 7750K .......... .......... .......... .......... .......... 65% 83.7M 0s 7800K .......... .......... .......... .......... .......... 65% 67.4M 0s 7850K .......... .......... .......... .......... .......... 65% 101M 0s 7900K .......... .......... .......... .......... .......... 66% 193M 0s 7950K .......... .......... .......... .......... .......... 66% 176M 0s 8000K .......... .......... .......... .......... .......... 67% 225M 0s 8050K .......... .......... .......... .......... .......... 67% 241M 0s 8100K .......... .......... .......... .......... .......... 68% 261M 0s 8150K .......... .......... .......... .......... .......... 68% 219M 0s 8200K .......... .......... .......... .......... .......... 68% 106M 0s 8250K .......... .......... .......... .......... .......... 69% 75.9M 0s 8300K .......... .......... .......... .......... .......... 69% 77.0M 0s 8350K .......... .......... .......... .......... .......... 70% 36.1M 0s 8400K .......... .......... .......... .......... .......... 70% 62.7M 0s 8450K .......... .......... .......... .......... .......... 70% 95.6M 0s 8500K .......... .......... .......... .......... .......... 71% 50.2M 0s 8550K .......... .......... .......... .......... .......... 71% 7.21M 0s 8600K .......... .......... .......... .......... .......... 72% 72.6M 0s 8650K .......... .......... .......... .......... .......... 72% 111M 0s 8700K .......... .......... .......... .......... .......... 73% 86.9M 0s 8750K .......... .......... .......... .......... .......... 73% 84.2M 0s 8800K .......... .......... .......... .......... .......... 73% 99.2M 0s 8850K .......... .......... .......... .......... .......... 74% 121M 0s 8900K .......... .......... .......... .......... .......... 74% 113M 0s 8950K .......... .......... .......... .......... .......... 75% 94.8M 0s 9000K .......... .......... .......... .......... .......... 75% 106M 0s 9050K .......... .......... .......... .......... .......... 75% 92.6M 0s 9100K .......... .......... .......... .......... .......... 76% 152M 0s 9150K .......... .......... .......... .......... .......... 76% 81.0M 0s 9200K .......... .......... .......... .......... .......... 77% 126M 0s 9250K .......... .......... .......... .......... .......... 77% 105M 0s 9300K .......... .......... .......... .......... .......... 78% 116M 0s 9350K .......... .......... .......... .......... .......... 78% 99.6M 0s 9400K .......... .......... .......... .......... .......... 78% 126M 0s 9450K .......... .......... .......... .......... .......... 79% 115M 0s 9500K .......... .......... .......... .......... .......... 79% 90.0M 0s 9550K .......... .......... .......... .......... .......... 80% 87.6M 0s 9600K .......... .......... .......... .......... .......... 80% 106M 0s 9650K .......... .......... .......... .......... .......... 80% 38.8M 0s 9700K .......... .......... .......... .......... .......... 81% 77.0M 0s 9750K .......... .......... .......... .......... .......... 81% 41.8M 0s 9800K .......... .......... .......... .......... .......... 82% 49.7M 0s 9850K .......... .......... .......... .......... .......... 82% 45.5M 0s 9900K .......... .......... .......... .......... .......... 83% 43.2M 0s 9950K .......... .......... .......... .......... .......... 83% 41.9M 0s 10000K .......... .......... .......... .......... .......... 83% 72.2M 0s 10050K .......... .......... .......... .......... .......... 84% 6.91M 0s 10100K .......... .......... .......... .......... .......... 84% 67.0M 0s 10150K .......... .......... .......... .......... .......... 85% 87.4M 0s 10200K .......... .......... .......... .......... .......... 85% 90.8M 0s 10250K .......... .......... .......... .......... .......... 85% 101M 0s 10300K .......... .......... .......... .......... .......... 86% 105M 0s 10350K .......... .......... .......... .......... .......... 86% 76.0M 0s 10400K .......... .......... .......... .......... .......... 87% 101M 0s 10450K .......... .......... .......... .......... .......... 87% 156M 0s 10500K .......... .......... .......... .......... .......... 88% 82.7M 0s 10550K .......... .......... .......... .......... .......... 88% 94.0M 0s 10600K .......... .......... .......... .......... .......... 88% 116M 0s 10650K .......... .......... .......... .......... .......... 89% 125M 0s 10700K .......... .......... .......... .......... .......... 89% 109M 0s 10750K .......... .......... .......... .......... .......... 90% 92.5M 0s 10800K .......... .......... .......... .......... .......... 90% 97.5M 0s 10850K .......... .......... .......... .......... .......... 90% 115M 0s 10900K .......... .......... .......... .......... .......... 91% 126M 0s 10950K .......... .......... .......... .......... .......... 91% 111M 0s 11000K .......... .......... .......... .......... .......... 92% 115M 0s 11050K .......... .......... .......... .......... .......... 92% 111M 0s 11100K .......... .......... .......... .......... .......... 93% 122M 0s 11150K .......... .......... .......... .......... .......... 93% 89.9M 0s 11200K .......... .......... .......... .......... .......... 93% 85.6M 0s 11250K .......... .......... .......... .......... .......... 94% 49.8M 0s 11300K .......... .......... .......... .......... .......... 94% 23.9M 0s 11350K .......... .......... .......... .......... .......... 95% 71.5M 0s 11400K .......... .......... .......... .......... .......... 95% 54.1M 0s 11450K .......... .......... .......... .......... .......... 95% 49.1M 0s 11500K .......... .......... .......... .......... .......... 96% 9.61M 0s 11550K .......... .......... .......... .......... .......... 96% 36.9M 0s 11600K .......... .......... .......... .......... .......... 97% 107M 0s 11650K .......... .......... .......... .......... .......... 97% 122M 0s 11700K .......... .......... .......... .......... .......... 98% 115M 0s 11750K .......... .......... .......... .......... .......... 98% 99.5M 0s 11800K .......... .......... .......... .......... .......... 98% 106M 0s 11850K .......... .......... .......... .......... .......... 99% 119M 0s 11900K .......... .......... .......... .......... .......... 99% 104M 0s 11950K .......... .......... .......... ... 100% 114M=0.3s 2020-10-27 12:48:57 (40.8 MB/s) - '/var/lib/xtesting/results/cnf_conformance/tools/helm/helm-v3.1.1-linux-amd64.tar.gz' saved [12270615/12270615] Global helm found. Version: v3.3.1 Local helm found. Version:  Global wget found. Version: 1.20.3 No Local wget version found Global curl found. Version: 7.67.0 No Local curl version found Global kubectl found. Version: 1.19+ No Local kubectl version found All prerequisites found. create_results_yml file exists?:false Successfully created directories for cnf-conformance Setup complete 2020-10-27 12:49:38,028 - functest_kubernetes.cnf_conformance.conformance - INFO - cnf-conformance cnf_setup cnf-config=cnf-conformance.yml Successfully created directories for cnf-conformance Successfully setup coredns 2020-10-27 12:52:13,844 - functest_kubernetes.cnf_conformance.conformance - INFO - cnf-conformance all install.go:158: [debug] Original chart version: "" install.go:175: [debug] CHART PATH: /root/.cache/helm/repository/coredns-1.13.4.tgz Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply create_results_yml file exists?:false ✔️ PASSED: No privileged containers Security final score: 5 of 5 ✔️ PASSED: Replicas increased to 3 📦📈 ✔️ PASSED: Replicas decreased to 1 📦📉 Scalability final score: 20 of 20 ✖️ FAILURE: IP addresses found ✔️ PASSED: Helm liveness probe found ✔️ PASSED: Helm readiness probe found ✔️ PASSED: CNF coredns-coredns Rolling Update Passed ✔️ PASSED: NodePort is not used ✔️ PASSED: No hard-coded IP addresses found in the runtime K8s configuration Configuration lifecycle final score: 24 of 25 PASSED (by default): No install script provided Successfully created directories for cnf-conformance ✔️ PASSED: Helm Chart cnfs/coredns/helm_chart/coredns Lint Passed ✔️ PASSED: Published Helm Chart Found ✔️ PASSED: Helm deploy successful Installability final score: 20 of 20 Hostname lookup for hub.docker.com failed: System error /home/pair/workspace/taylor/qa/jun8/lib/halite/src/halite/client.cr:155:7 in 'perform' /home/pair/workspace/taylor/qa/jun8/lib/halite/src/halite/client.cr:115:9 in 'request' /home/pair/workspace/taylor/qa/jun8/lib/halite/src/halite/chainable.cr:485:7 in '->' /home/pair/workspace/taylor/qa/jun8/src/tasks/utils/utils.cr:255:3 in 'task_runner' /home/pair/workspace/taylor/qa/jun8/src/tasks/microservice.cr:93:3 in '->' /home/pair/workspace/taylor/qa/jun8/lib/sam/src/sam/task.cr:39:29 in 'call' /home/pair/workspace/taylor/qa/jun8/lib/sam/src/sam/task.cr:39:29 in 'call' /home/pair/workspace/taylor/qa/jun8/lib/sam/src/sam.cr:43:5 in '__crystal_main' /usr/local/depot/crystal-0.33.0-1/share/crystal/src/crystal/main.cr:106:5 in 'main' ??? ??? ✔️ PASSED: CNF had a reasonable startup time 🚀 Microservice final score: 5 of 10 Final score: 74 of 80 Results have been saved to cnf-conformance-results-20201027-125213-836.yml 2020-10-27 12:52:13,863 - functest_kubernetes.cnf_conformance.conformance - INFO - +-------------------------------------------------------------+----------------+ | NAME | STATUS | +-------------------------------------------------------------+----------------+ | privileged | passed | | increase_capacity | passed | | decrease_capacity | passed | | ip_addresses | failed | | liveness | passed | | readiness | passed | | rolling_update | passed | | nodeport_not_used | passed | | hardcoded_ip_addresses_in_k8s_runtime_configuration | passed | | install_script_helm | passed | | helm_chart_valid | passed | | helm_chart_published | passed | | helm_deploy | passed | | reasonable_startup_time | passed | +-------------------------------------------------------------+----------------+ 2020-10-27 12:52:13,864 - xtesting.ci.run_tests - INFO - Test result: +-------------------------+------------------+------------------+----------------+ | TEST CASE | PROJECT | DURATION | RESULT | +-------------------------+------------------+------------------+----------------+ | cnf_conformance | functest | 03:21 | PASS | +-------------------------+------------------+------------------+----------------+ 2020-10-27 12:52:18,862 - functest_kubernetes.cnf_conformance.conformance - INFO - cnf-conformance cnf_cleanup cnf-config=cnf-conformance.yml Successfully cleaned up coredns