Oct 26 13:50:17 ctl01 rsyslogd: [origin software="rsyslogd" swVersion="8.16.0" x-pid="1185" x-info="http://www.rsyslog.com"] start
Oct 26 13:50:17 ctl01 systemd-modules-load[460]: Inserted module 'iscsi_tcp'
Oct 26 13:50:17 ctl01 loadkeys[458]: Loading /etc/console-setup/cached.kmap.gz
Oct 26 13:50:17 ctl01 systemd-modules-load[460]: Inserted module 'ib_iser'
Oct 26 13:50:17 ctl01 systemd[1]: Started Load Kernel Modules.
Oct 26 13:50:17 ctl01 systemd[1]: Starting Apply Kernel Variables...
Oct 26 13:50:17 ctl01 systemd[1]: Mounting Configuration File System...
Oct 26 13:50:17 ctl01 systemd[1]: Mounting FUSE Control File System...
Oct 26 13:50:17 ctl01 systemd[1]: Started LVM2 metadata daemon.
Oct 26 13:50:17 ctl01 systemd[1]: Starting Flush Journal to Persistent Storage...
Oct 26 13:50:17 ctl01 systemd[1]: Starting Load/Save Random Seed...
Oct 26 13:50:17 ctl01 systemd[1]: Starting udev Coldplug all Devices...
Oct 26 13:50:17 ctl01 systemd[1]: Starting Create Static Device Nodes in /dev...
Oct 26 13:50:17 ctl01 systemd[1]: Mounted Configuration File System.
Oct 26 13:50:17 ctl01 systemd[1]: Mounted FUSE Control File System.
Oct 26 13:50:17 ctl01 systemd[1]: Started Apply Kernel Variables.
Oct 26 13:50:17 ctl01 systemd[1]: Started Load/Save Random Seed.
Oct 26 13:50:17 ctl01 systemd[1]: Started Create Static Device Nodes in /dev.
Oct 26 13:50:17 ctl01 systemd[1]: Starting udev Kernel Device Manager...
Oct 26 13:50:17 ctl01 systemd[1]: Started Flush Journal to Persistent Storage.
Oct 26 13:50:17 ctl01 rsyslogd-2222: command 'KLogPermitNonKernelFacility' is currently not permitted - did you already set it via a RainerScript command (v6+ config)? [v8.16.0 try http://www.rsyslog.com/e/2222 ]
Oct 26 13:50:17 ctl01 rsyslogd: rsyslogd's groupid changed to 108
Oct 26 13:50:17 ctl01 rsyslogd: rsyslogd's userid changed to 104
Oct 26 13:50:17 ctl01 systemd[1]: Started udev Coldplug all Devices.
Oct 26 13:50:17 ctl01 systemd[1]: Started udev Kernel Device Manager.
Oct 26 13:50:17 ctl01 systemd[1]: Started Dispatch Password Requests to Console Directory Watch.
Oct 26 13:50:17 ctl01 systemd[1]: Found device /dev/ttyS0.
Oct 26 13:50:17 ctl01 systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.
Oct 26 13:50:17 ctl01 systemd[1]: Started Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling.
Oct 26 13:50:17 ctl01 systemd[1]: Reached target Local File Systems (Pre).
Oct 26 13:50:17 ctl01 systemd[1]: Reached target Local File Systems.
Oct 26 13:50:17 ctl01 systemd[1]: Starting Tell Plymouth To Write Out Runtime Data...
Oct 26 13:50:17 ctl01 rsyslogd-2039: Could not open output pipe '/dev/xconsole':: No such file or directory [v8.16.0 try http://www.rsyslog.com/e/2039 ]
Oct 26 13:50:17 ctl01 rsyslogd-2007: action 'action 11' suspended, next retry is Sat Oct 26 13:50:47 2019 [v8.16.0 try http://www.rsyslog.com/e/2007 ]
Oct 26 13:50:17 ctl01 systemd[1]: Starting Commit a transient machine-id on disk...
Oct 26 13:50:17 ctl01 systemd[1]: Starting Set console font and keymap...
Oct 26 13:50:17 ctl01 systemd[1]: Starting Initial cloud-init job (pre-networking)...
Oct 26 13:50:17 ctl01 systemd[1]: Starting Create Volatile Files and Directories...
Oct 26 13:50:17 ctl01 systemd[1]: Starting LSB: AppArmor initialization...
Oct 26 13:50:17 ctl01 systemd-tmpfiles[693]: [/usr/lib/tmpfiles.d/var.conf:14] Duplicate line for path "/var/log", ignoring.
Oct 26 13:50:17 ctl01 systemd[1]: Started Tell Plymouth To Write Out Runtime Data.
Oct 26 13:50:17 ctl01 systemd[1]: Started Create Volatile Files and Directories.
Oct 26 13:50:17 ctl01 systemd[1]: Starting Network Time Synchronization...
Oct 26 13:50:17 ctl01 systemd[1]: Starting Update UTMP about System Boot/Shutdown...
Oct 26 13:50:17 ctl01 systemd[1]: Started Commit a transient machine-id on disk.
Oct 26 13:50:17 ctl01 systemd[1]: Started Network Time Synchronization.
Oct 26 13:50:17 ctl01 systemd[1]: Reached target System Time Synchronized.
Oct 26 13:50:17 ctl01 apparmor[695]:  * Starting AppArmor profiles
Oct 26 13:50:17 ctl01 systemd[1]: Started Update UTMP about System Boot/Shutdown.
Oct 26 13:50:17 ctl01 systemd[1]: Started Set console font and keymap.
Oct 26 13:50:17 ctl01 systemd[1]: Created slice system-getty.slice.
Oct 26 13:50:17 ctl01 apparmor[695]: Skipping profile in /etc/apparmor.d/disable: usr.sbin.rsyslogd
Oct 26 13:50:17 ctl01 apparmor[695]:    ...done.
Oct 26 13:50:17 ctl01 systemd[1]: Started LSB: AppArmor initialization.
Oct 26 13:50:17 ctl01 cloud-init[689]: Cloud-init v. 19.2-36-g059d049c-0ubuntu2~16.04.1 running 'init-local' at Sat, 26 Oct 2019 13:50:11 +0000. Up 5.09 seconds.
Oct 26 13:50:17 ctl01 kernel: [    0.000000] Linux version 4.15.0-66-generic (buildd@lgw01-amd64-050) (gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.10)) #75~16.04.1-Ubuntu SMP Tue Oct 1 14:01:08 UTC 2019 (Ubuntu 4.15.0-66.75~16.04.1-generic 4.15.18)
Oct 26 13:50:17 ctl01 kernel: [    0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-4.15.0-66-generic root=UUID=3dc7f99a-1e06-428f-86bf-f1f8932b4515 ro console=tty1 console=ttyS0
Oct 26 13:50:17 ctl01 kernel: [    0.000000] KERNEL supported cpus:
Oct 26 13:50:17 ctl01 kernel: [    0.000000]   Intel GenuineIntel
Oct 26 13:50:17 ctl01 kernel: [    0.000000]   AMD AuthenticAMD
Oct 26 13:50:17 ctl01 kernel: [    0.000000]   Centaur CentaurHauls
Oct 26 13:50:17 ctl01 kernel: [    0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
Oct 26 13:50:17 ctl01 kernel: [    0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
Oct 26 13:50:17 ctl01 kernel: [    0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
Oct 26 13:50:17 ctl01 kernel: [    0.000000] x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256
Oct 26 13:50:17 ctl01 kernel: [    0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format.
Oct 26 13:50:17 ctl01 kernel: [    0.000000] e820: BIOS-provided physical RAM map:
Oct 26 13:50:17 ctl01 kernel: [    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
Oct 26 13:50:17 ctl01 kernel: [    0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
Oct 26 13:50:17 ctl01 kernel: [    0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
Oct 26 13:50:17 ctl01 kernel: [    0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffdefff] usable
Oct 26 13:50:17 ctl01 kernel: [    0.000000] BIOS-e820: [mem 0x00000000bffdf000-0x00000000bfffffff] reserved
Oct 26 13:50:17 ctl01 kernel: [    0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved
Oct 26 13:50:17 ctl01 kernel: [    0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
Oct 26 13:50:17 ctl01 kernel: [    0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000003bfffffff] usable
Oct 26 13:50:17 ctl01 kernel: [    0.000000] NX (Execute Disable) protection: active
Oct 26 13:50:17 ctl01 kernel: [    0.000000] SMBIOS 2.8 present.
Oct 26 13:50:17 ctl01 kernel: [    0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Ubuntu-1.8.2-1ubuntu1 04/01/2014
Oct 26 13:50:17 ctl01 kernel: [    0.000000] Hypervisor detected: KVM
Oct 26 13:50:17 ctl01 kernel: [    0.000000] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
Oct 26 13:50:17 ctl01 kernel: [    0.000000] e820: remove [mem 0x000a0000-0x000fffff] usable
Oct 26 13:50:17 ctl01 kernel: [    0.000000] e820: last_pfn = 0x3c0000 max_arch_pfn = 0x400000000
Oct 26 13:50:17 ctl01 kernel: [    0.000000] MTRR default type: write-back
Oct 26 13:50:17 ctl01 kernel: [    0.000000] MTRR fixed ranges enabled:
Oct 26 13:50:17 ctl01 kernel: [    0.000000]   00000-9FFFF write-back
Oct 26 13:50:17 ctl01 kernel: [    0.000000]   A0000-BFFFF uncachable
Oct 26 13:50:17 ctl01 kernel: [    0.000000]   C0000-FFFFF write-protect
Oct 26 13:50:17 ctl01 kernel: [    0.000000] MTRR variable ranges enabled:
Oct 26 13:50:17 ctl01 kernel: [    0.000000]   0 base 00C0000000 mask FFC0000000 uncachable
Oct 26 13:50:17 ctl01 kernel: [    0.000000]   1 disabled
Oct 26 13:50:17 ctl01 kernel: [    0.000000]   2 disabled
Oct 26 13:50:17 ctl01 kernel: [    0.000000]   3 disabled
Oct 26 13:50:17 ctl01 kernel: [    0.000000]   4 disabled
Oct 26 13:50:17 ctl01 kernel: [    0.000000]   5 disabled
Oct 26 13:50:17 ctl01 kernel: [    0.000000]   6 disabled
Oct 26 13:50:17 ctl01 kernel: [    0.000000]   7 disabled
Oct 26 13:50:17 ctl01 kernel: [    0.000000] x86/PAT: Configuration [0-7]: WB  WC  UC- UC  WB  WP  UC- WT  
Oct 26 13:50:17 ctl01 kernel: [    0.000000] e820: last_pfn = 0xbffdf max_arch_pfn = 0x400000000
Oct 26 13:50:17 ctl01 kernel: [    0.000000] found SMP MP-table at [mem 0x000f6560-0x000f656f]
Oct 26 13:50:17 ctl01 kernel: [    0.000000] Scanning 1 areas for low memory corruption
Oct 26 13:50:17 ctl01 kernel: [    0.000000] Using GB pages for direct mapping
Oct 26 13:50:17 ctl01 kernel: [    0.000000] BRK [0x37173f000, 0x37173ffff] PGTABLE
Oct 26 13:50:17 ctl01 kernel: [    0.000000] BRK [0x371740000, 0x371740fff] PGTABLE
Oct 26 13:50:17 ctl01 kernel: [    0.000000] BRK [0x371741000, 0x371741fff] PGTABLE
Oct 26 13:50:17 ctl01 kernel: [    0.000000] BRK [0x371742000, 0x371742fff] PGTABLE
Oct 26 13:50:17 ctl01 kernel: [    0.000000] BRK [0x371743000, 0x371743fff] PGTABLE
Oct 26 13:50:17 ctl01 kernel: [    0.000000] BRK [0x371744000, 0x371744fff] PGTABLE
Oct 26 13:50:17 ctl01 kernel: [    0.000000] RAMDISK: [mem 0x31726000-0x34b8afff]
Oct 26 13:50:17 ctl01 kernel: [    0.000000] ACPI: Early table checksum verification disabled
Oct 26 13:50:17 ctl01 kernel: [    0.000000] ACPI: RSDP 0x00000000000F6510 000014 (v00 BOCHS )
Oct 26 13:50:17 ctl01 kernel: [    0.000000] ACPI: RSDT 0x00000000BFFE14CC 000030 (v01 BOCHS  BXPCRSDT 00000001 BXPC 00000001)
Oct 26 13:50:17 ctl01 kernel: [    0.000000] ACPI: FACP 0x00000000BFFE0854 000074 (v01 BOCHS  BXPCFACP 00000001 BXPC 00000001)
Oct 26 13:50:17 ctl01 kernel: [    0.000000] ACPI: DSDT 0x00000000BFFDFC80 000BD4 (v01 BOCHS  BXPCDSDT 00000001 BXPC 00000001)
Oct 26 13:50:17 ctl01 kernel: [    0.000000] ACPI: FACS 0x00000000BFFDFC40 000040
Oct 26 13:50:17 ctl01 kernel: [    0.000000] ACPI: SSDT 0x00000000BFFE08C8 000B54 (v01 BOCHS  BXPCSSDT 00000001 BXPC 00000001)
Oct 26 13:50:17 ctl01 kernel: [    0.000000] ACPI: APIC 0x00000000BFFE141C 0000B0 (v01 BOCHS  BXPCAPIC 00000001 BXPC 00000001)
Oct 26 13:50:17 ctl01 kernel: [    0.000000] ACPI: Local APIC address 0xfee00000
Oct 26 13:50:17 ctl01 kernel: [    0.000000] No NUMA configuration found
Oct 26 13:50:17 ctl01 kernel: [    0.000000] Faking a node at [mem 0x0000000000000000-0x00000003bfffffff]
Oct 26 13:50:17 ctl01 kernel: [    0.000000] NODE_DATA(0) allocated [mem 0x3bffd5000-0x3bfffffff]
Oct 26 13:50:17 ctl01 kernel: [    0.000000] kvm-clock: cpu 0, msr 3:bff54001, primary cpu clock
Oct 26 13:50:17 ctl01 kernel: [    0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00
Oct 26 13:50:17 ctl01 kernel: [    0.000000] kvm-clock: using sched offset of 1684222567 cycles
Oct 26 13:50:17 ctl01 kernel: [    0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns
Oct 26 13:50:17 ctl01 kernel: [    0.000000] Zone ranges:
Oct 26 13:50:17 ctl01 kernel: [    0.000000]   DMA      [mem 0x0000000000001000-0x0000000000ffffff]
Oct 26 13:50:17 ctl01 kernel: [    0.000000]   DMA32    [mem 0x0000000001000000-0x00000000ffffffff]
Oct 26 13:50:17 ctl01 kernel: [    0.000000]   Normal   [mem 0x0000000100000000-0x00000003bfffffff]
Oct 26 13:50:17 ctl01 kernel: [    0.000000]   Device   empty
Oct 26 13:50:17 ctl01 kernel: [    0.000000] Movable zone start for each node
Oct 26 13:50:17 ctl01 kernel: [    0.000000] Early memory node ranges
Oct 26 13:50:17 ctl01 kernel: [    0.000000]   node   0: [mem 0x0000000000001000-0x000000000009efff]
Oct 26 13:50:17 ctl01 kernel: [    0.000000]   node   0: [mem 0x0000000000100000-0x00000000bffdefff]
Oct 26 13:50:17 ctl01 kernel: [    0.000000]   node   0: [mem 0x0000000100000000-0x00000003bfffffff]
Oct 26 13:50:17 ctl01 kernel: [    0.000000] Reserved but unavailable: 98 pages
Oct 26 13:50:17 ctl01 kernel: [    0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x00000003bfffffff]
Oct 26 13:50:17 ctl01 kernel: [    0.000000] On node 0 totalpages: 3669885
Oct 26 13:50:17 ctl01 kernel: [    0.000000]   DMA zone: 64 pages used for memmap
Oct 26 13:50:17 ctl01 kernel: [    0.000000]   DMA zone: 21 pages reserved
Oct 26 13:50:17 ctl01 kernel: [    0.000000]   DMA zone: 3998 pages, LIFO batch:0
Oct 26 13:50:17 ctl01 kernel: [    0.000000]   DMA32 zone: 12224 pages used for memmap
Oct 26 13:50:17 ctl01 kernel: [    0.000000]   DMA32 zone: 782303 pages, LIFO batch:31
Oct 26 13:50:17 ctl01 kernel: [    0.000000]   Normal zone: 45056 pages used for memmap
Oct 26 13:50:17 ctl01 kernel: [    0.000000]   Normal zone: 2883584 pages, LIFO batch:31
Oct 26 13:50:17 ctl01 kernel: [    0.000000] ACPI: PM-Timer IO Port: 0x608
Oct 26 13:50:17 ctl01 kernel: [    0.000000] ACPI: Local APIC address 0xfee00000
Oct 26 13:50:17 ctl01 kernel: [    0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
Oct 26 13:50:17 ctl01 kernel: [    0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23
Oct 26 13:50:17 ctl01 kernel: [    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
Oct 26 13:50:17 ctl01 kernel: [    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
Oct 26 13:50:17 ctl01 kernel: [    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
Oct 26 13:50:17 ctl01 kernel: [    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
Oct 26 13:50:17 ctl01 kernel: [    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
Oct 26 13:50:17 ctl01 kernel: [    0.000000] ACPI: IRQ0 used by override.
Oct 26 13:50:17 ctl01 kernel: [    0.000000] ACPI: IRQ5 used by override.
Oct 26 13:50:17 ctl01 kernel: [    0.000000] ACPI: IRQ9 used by override.
Oct 26 13:50:17 ctl01 kernel: [    0.000000] ACPI: IRQ10 used by override.
Oct 26 13:50:17 ctl01 kernel: [    0.000000] ACPI: IRQ11 used by override.
Oct 26 13:50:17 ctl01 kernel: [    0.000000] Using ACPI (MADT) for SMP configuration information
Oct 26 13:50:17 ctl01 kernel: [    0.000000] smpboot: Allowing 8 CPUs, 0 hotplug CPUs
Oct 26 13:50:17 ctl01 kernel: [    0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff]
Oct 26 13:50:17 ctl01 kernel: [    0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff]
Oct 26 13:50:17 ctl01 kernel: [    0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff]
Oct 26 13:50:17 ctl01 kernel: [    0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff]
Oct 26 13:50:17 ctl01 kernel: [    0.000000] PM: Registered nosave memory: [mem 0xbffdf000-0xbfffffff]
Oct 26 13:50:17 ctl01 kernel: [    0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff]
Oct 26 13:50:17 ctl01 kernel: [    0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff]
Oct 26 13:50:17 ctl01 kernel: [    0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff]
Oct 26 13:50:17 ctl01 kernel: [    0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff]
Oct 26 13:50:17 ctl01 kernel: [    0.000000] e820: [mem 0xc0000000-0xfeffbfff] available for PCI devices
Oct 26 13:50:17 ctl01 kernel: [    0.000000] Booting paravirtualized kernel on KVM
Oct 26 13:50:17 ctl01 kernel: [    0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns
Oct 26 13:50:17 ctl01 kernel: [    0.000000] random: get_random_bytes called from start_kernel+0x99/0x51b with crng_init=0
Oct 26 13:50:17 ctl01 kernel: [    0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:8 nr_cpu_ids:8 nr_node_ids:1
Oct 26 13:50:17 ctl01 kernel: [    0.000000] percpu: Embedded 46 pages/cpu s151552 r8192 d28672 u262144
Oct 26 13:50:17 ctl01 kernel: [    0.000000] pcpu-alloc: s151552 r8192 d28672 u262144 alloc=1*2097152
Oct 26 13:50:17 ctl01 kernel: [    0.000000] pcpu-alloc: [0] 0 1 2 3 4 5 6 7 
Oct 26 13:50:17 ctl01 kernel: [    0.000000] KVM setup async PF for cpu 0
Oct 26 13:50:17 ctl01 kernel: [    0.000000] kvm-stealtime: cpu 0, msr 3bfc24040
Oct 26 13:50:17 ctl01 kernel: [    0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes)
Oct 26 13:50:17 ctl01 kernel: [    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 3612520
Oct 26 13:50:17 ctl01 kernel: [    0.000000] Policy zone: Normal
Oct 26 13:50:17 ctl01 kernel: [    0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-4.15.0-66-generic root=UUID=3dc7f99a-1e06-428f-86bf-f1f8932b4515 ro console=tty1 console=ttyS0
Oct 26 13:50:17 ctl01 kernel: [    0.000000] Calgary: detecting Calgary via BIOS EBDA area
Oct 26 13:50:17 ctl01 kernel: [    0.000000] Calgary: Unable to locate Rio Grande table in EBDA - bailing!
Oct 26 13:50:17 ctl01 kernel: [    0.000000] Memory: 14300308K/14679540K available (12300K kernel code, 2480K rwdata, 4208K rodata, 2428K init, 2388K bss, 379232K reserved, 0K cma-reserved)
Oct 26 13:50:17 ctl01 kernel: [    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=8, Nodes=1
Oct 26 13:50:17 ctl01 kernel: [    0.000000] Kernel/User page tables isolation: enabled
Oct 26 13:50:17 ctl01 kernel: [    0.000000] ftrace: allocating 39287 entries in 154 pages
Oct 26 13:50:17 ctl01 kernel: [    0.004000] Hierarchical RCU implementation.
Oct 26 13:50:17 ctl01 kernel: [    0.004000] 	RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=8.
Oct 26 13:50:17 ctl01 kernel: [    0.004000] 	Tasks RCU enabled.
Oct 26 13:50:17 ctl01 kernel: [    0.004000] RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=8
Oct 26 13:50:17 ctl01 kernel: [    0.004000] NR_IRQS: 524544, nr_irqs: 488, preallocated irqs: 16
Oct 26 13:50:17 ctl01 kernel: [    0.004000] Console: colour VGA+ 80x25
Oct 26 13:50:17 ctl01 kernel: [    0.004000] console [tty1] enabled
Oct 26 13:50:17 ctl01 kernel: [    0.004000] console [ttyS0] enabled
Oct 26 13:50:17 ctl01 kernel: [    0.004000] ACPI: Core revision 20170831
Oct 26 13:50:17 ctl01 kernel: [    0.004000] ACPI: 2 ACPI AML tables successfully acquired and loaded
Oct 26 13:50:17 ctl01 kernel: [    0.004005] APIC: Switch to symmetric I/O mode setup
Oct 26 13:50:17 ctl01 kernel: [    0.005514] x2apic enabled
Oct 26 13:50:17 ctl01 kernel: [    0.006587] Switched APIC routing to physical x2apic.
Oct 26 13:50:17 ctl01 kernel: [    0.009217] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
Oct 26 13:50:17 ctl01 kernel: [    0.010656] tsc: Detected 2799.994 MHz processor
Oct 26 13:50:17 ctl01 kernel: [    0.011788] Calibrating delay loop (skipped) preset value.. 5599.98 BogoMIPS (lpj=11199976)
Oct 26 13:50:17 ctl01 kernel: [    0.012002] pid_max: default: 32768 minimum: 301
Oct 26 13:50:17 ctl01 kernel: [    0.013146] Security Framework initialized
Oct 26 13:50:17 ctl01 kernel: [    0.014160] Yama: becoming mindful.
Oct 26 13:50:17 ctl01 kernel: [    0.016027] AppArmor: AppArmor initialized
Oct 26 13:50:17 ctl01 kernel: [    0.022838] Dentry cache hash table entries: 2097152 (order: 12, 16777216 bytes)
Oct 26 13:50:17 ctl01 kernel: [    0.026700] Inode-cache hash table entries: 1048576 (order: 11, 8388608 bytes)
Oct 26 13:50:17 ctl01 kernel: [    0.028035] Mount-cache hash table entries: 32768 (order: 6, 262144 bytes)
Oct 26 13:50:17 ctl01 kernel: [    0.030129] Mountpoint-cache hash table entries: 32768 (order: 6, 262144 bytes)
Oct 26 13:50:17 ctl01 kernel: [    0.032308] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0
Oct 26 13:50:17 ctl01 kernel: [    0.033581] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0
Oct 26 13:50:17 ctl01 kernel: [    0.036004] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization
Oct 26 13:50:17 ctl01 kernel: [    0.037979] Spectre V2 : Mitigation: Full generic retpoline
Oct 26 13:50:17 ctl01 kernel: [    0.039288] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch
Oct 26 13:50:17 ctl01 kernel: [    0.040002] Spectre V2 : Enabling Restricted Speculation for firmware calls
Oct 26 13:50:17 ctl01 kernel: [    0.041603] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier
Oct 26 13:50:17 ctl01 kernel: [    0.044004] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl and seccomp
Oct 26 13:50:17 ctl01 kernel: [    0.048045] MDS: Mitigation: Clear CPU buffers
Oct 26 13:50:17 ctl01 kernel: [    0.055687] Freeing SMP alternatives memory: 36K
Oct 26 13:50:17 ctl01 kernel: [    0.058124] TSC deadline timer enabled
Oct 26 13:50:17 ctl01 kernel: [    0.058128] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz (family: 0x6, model: 0x3e, stepping: 0x4)
Oct 26 13:50:17 ctl01 systemd[1]: Started Initial cloud-init job (pre-networking).
Oct 26 13:50:17 ctl01 kernel: [    0.060000] Performance Events: IvyBridge events, Intel PMU driver.
Oct 26 13:50:17 ctl01 kernel: [    0.060000] ... version:                2
Oct 26 13:50:17 ctl01 kernel: [    0.060005] ... bit width:              48
Oct 26 13:50:17 ctl01 kernel: [    0.061008] ... generic registers:      4
Oct 26 13:50:17 ctl01 kernel: [    0.061992] ... value mask:             0000ffffffffffff
Oct 26 13:50:17 ctl01 systemd[1]: Reached target Network (Pre).
Oct 26 13:50:17 ctl01 kernel: [    0.063261] ... max period:             000000007fffffff
Oct 26 13:50:17 ctl01 systemd[1]: Starting Raise network interfaces...
Oct 26 13:50:17 ctl01 kernel: [    0.064004] ... fixed-purpose events:   3
Oct 26 13:50:17 ctl01 kernel: [    0.064987] ... event mask:             000000070000000f
Oct 26 13:50:17 ctl01 kernel: [    0.066279] Hierarchical SRCU implementation.
Oct 26 13:50:17 ctl01 dhclient[938]: Internet Systems Consortium DHCP Client 4.3.3
Oct 26 13:50:17 ctl01 kernel: [    0.068086] smp: Bringing up secondary CPUs ...
Oct 26 13:50:17 ctl01 kernel: [    0.069285] x86: Booting SMP configuration:
Oct 26 13:50:17 ctl01 kernel: [    0.070319] .... node  #0, CPUs:      #1
Oct 26 13:50:17 ctl01 kernel: [    0.004000] kvm-clock: cpu 1, msr 3:bff54041, secondary cpu clock
Oct 26 13:50:17 ctl01 ifup[837]: Internet Systems Consortium DHCP Client 4.3.3
Oct 26 13:50:17 ctl01 kernel: [    0.076025] KVM setup async PF for cpu 1
Oct 26 13:50:17 ctl01 kernel: [    0.076973] kvm-stealtime: cpu 1, msr 3bfc64040
Oct 26 13:50:17 ctl01 kernel: [    0.078082]  #2
Oct 26 13:50:17 ctl01 kernel: [    0.004000] kvm-clock: cpu 2, msr 3:bff54081, secondary cpu clock
Oct 26 13:50:17 ctl01 ifup[837]: Copyright 2004-2015 Internet Systems Consortium.
Oct 26 13:50:17 ctl01 kernel: [    0.080023] KVM setup async PF for cpu 2
Oct 26 13:50:17 ctl01 kernel: [    0.080972] kvm-stealtime: cpu 2, msr 3bfca4040
Oct 26 13:50:17 ctl01 kernel: [    0.082072]  #3
Oct 26 13:50:17 ctl01 ifup[837]: All rights reserved.
Oct 26 13:50:17 ctl01 ifup[837]: For info, please visit https://www.isc.org/software/dhcp/
Oct 26 13:50:17 ctl01 dhclient[938]: Copyright 2004-2015 Internet Systems Consortium.
Oct 26 13:50:17 ctl01 dhclient[938]: All rights reserved.
Oct 26 13:50:17 ctl01 dhclient[938]: For info, please visit https://www.isc.org/software/dhcp/
Oct 26 13:50:17 ctl01 dhclient[938]: 
Oct 26 13:50:17 ctl01 dhclient[938]: Listening on LPF/ens3/52:54:00:d1:44:f2
Oct 26 13:50:17 ctl01 ifup[837]: Listening on LPF/ens3/52:54:00:d1:44:f2
Oct 26 13:50:17 ctl01 ifup[837]: Sending on   LPF/ens3/52:54:00:d1:44:f2
Oct 26 13:50:17 ctl01 ifup[837]: Sending on   Socket/fallback
Oct 26 13:50:17 ctl01 ifup[837]: DHCPDISCOVER on ens3 to 255.255.255.255 port 67 interval 3 (xid=0xc9418c5b)
Oct 26 13:50:17 ctl01 dhclient[938]: Sending on   LPF/ens3/52:54:00:d1:44:f2
Oct 26 13:50:17 ctl01 dhclient[938]: Sending on   Socket/fallback
Oct 26 13:50:17 ctl01 dhclient[938]: DHCPDISCOVER on ens3 to 255.255.255.255 port 67 interval 3 (xid=0xc9418c5b)
Oct 26 13:50:17 ctl01 dhclient[938]: DHCPREQUEST of 192.168.11.155 on ens3 to 255.255.255.255 port 67 (xid=0x5b8c41c9)
Oct 26 13:50:17 ctl01 ifup[837]: DHCPREQUEST of 192.168.11.155 on ens3 to 255.255.255.255 port 67 (xid=0x5b8c41c9)
Oct 26 13:50:17 ctl01 ifup[837]: DHCPOFFER of 192.168.11.155 from 192.168.11.3
Oct 26 13:50:17 ctl01 dhclient[938]: DHCPOFFER of 192.168.11.155 from 192.168.11.3
Oct 26 13:50:17 ctl01 ifup[837]: DHCPACK of 192.168.11.155 from 192.168.11.3
Oct 26 13:50:17 ctl01 dhclient[938]: DHCPACK of 192.168.11.155 from 192.168.11.3
Oct 26 13:50:17 ctl01 dhclient[938]: bound to 192.168.11.155 -- renewal in 1752 seconds.
Oct 26 13:50:17 ctl01 ifup[837]: bound to 192.168.11.155 -- renewal in 1752 seconds.
Oct 26 13:50:17 ctl01 systemd[1]: Started Raise network interfaces.
Oct 26 13:50:17 ctl01 systemd[1]: Reached target Network.
Oct 26 13:50:17 ctl01 systemd[1]: Starting Initial cloud-init job (metadata service crawler)...
Oct 26 13:50:17 ctl01 kernel: [    0.004000] kvm-clock: cpu 3, msr 3:bff540c1, secondary cpu clock
Oct 26 13:50:17 ctl01 kernel: [    0.084022] KVM setup async PF for cpu 3
Oct 26 13:50:17 ctl01 kernel: [    0.084971] kvm-stealtime: cpu 3, msr 3bfce4040
Oct 26 13:50:17 ctl01 kernel: [    0.086073]  #4
Oct 26 13:50:17 ctl01 kernel: [    0.004000] kvm-clock: cpu 4, msr 3:bff54101, secondary cpu clock
Oct 26 13:50:17 ctl01 kernel: [    0.088021] KVM setup async PF for cpu 4
Oct 26 13:50:17 ctl01 kernel: [    0.088965] kvm-stealtime: cpu 4, msr 3bfd24040
Oct 26 13:50:17 ctl01 kernel: [    0.090055]  #5
Oct 26 13:50:17 ctl01 kernel: [    0.004000] kvm-clock: cpu 5, msr 3:bff54141, secondary cpu clock
Oct 26 13:50:17 ctl01 kernel: [    0.092023] KVM setup async PF for cpu 5
Oct 26 13:50:17 ctl01 kernel: [    0.092968] kvm-stealtime: cpu 5, msr 3bfd64040
Oct 26 13:50:17 ctl01 kernel: [    0.094073]  #6
Oct 26 13:50:17 ctl01 kernel: [    0.004000] kvm-clock: cpu 6, msr 3:bff54181, secondary cpu clock
Oct 26 13:50:17 ctl01 kernel: [    0.096021] KVM setup async PF for cpu 6
Oct 26 13:50:17 ctl01 kernel: [    0.096973] kvm-stealtime: cpu 6, msr 3bfda4040
Oct 26 13:50:17 ctl01 kernel: [    0.098066]  #7
Oct 26 13:50:17 ctl01 kernel: [    0.004000] kvm-clock: cpu 7, msr 3:bff541c1, secondary cpu clock
Oct 26 13:50:17 ctl01 kernel: [    0.104021] KVM setup async PF for cpu 7
Oct 26 13:50:17 ctl01 kernel: [    0.104970] kvm-stealtime: cpu 7, msr 3bfde4040
Oct 26 13:50:17 ctl01 kernel: [    0.106075] smp: Brought up 1 node, 8 CPUs
Oct 26 13:50:17 ctl01 kernel: [    0.108006] smpboot: Max logical packages: 8
Oct 26 13:50:17 ctl01 kernel: [    0.109094] smpboot: Total of 8 processors activated (44799.90 BogoMIPS)
Oct 26 13:50:17 ctl01 kernel: [    0.111454] devtmpfs: initialized
Oct 26 13:50:17 ctl01 kernel: [    0.112057] x86/mm: Memory block size: 128MB
Oct 26 13:50:17 ctl01 kernel: [    0.114114] evm: security.selinux
Oct 26 13:50:17 ctl01 kernel: [    0.114999] evm: security.SMACK64
Oct 26 13:50:17 ctl01 kernel: [    0.115878] evm: security.SMACK64EXEC
Oct 26 13:50:17 ctl01 kernel: [    0.116006] evm: security.SMACK64TRANSMUTE
Oct 26 13:50:17 ctl01 kernel: [    0.117044] evm: security.SMACK64MMAP
Oct 26 13:50:17 ctl01 kernel: [    0.118002] evm: security.apparmor
Oct 26 13:50:17 ctl01 kernel: [    0.118892] evm: security.ima
Oct 26 13:50:17 ctl01 kernel: [    0.119690] evm: security.capability
Oct 26 13:50:17 ctl01 kernel: [    0.120116] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
Oct 26 13:50:17 ctl01 kernel: [    0.122360] futex hash table entries: 2048 (order: 5, 131072 bytes)
Oct 26 13:50:17 ctl01 kernel: [    0.124092] pinctrl core: initialized pinctrl subsystem
Oct 26 13:50:17 ctl01 kernel: [    0.125551] RTC time: 13:50:06, date: 10/26/19
Oct 26 13:50:17 ctl01 kernel: [    0.127523] NET: Registered protocol family 16
Oct 26 13:50:17 ctl01 kernel: [    0.128081] audit: initializing netlink subsys (disabled)
Oct 26 13:50:17 ctl01 kernel: [    0.129434] audit: type=2000 audit(1572097806.927:1): state=initialized audit_enabled=0 res=1
Oct 26 13:50:17 ctl01 kernel: [    0.132009] cpuidle: using governor ladder
Oct 26 13:50:17 ctl01 kernel: [    0.133089] cpuidle: using governor menu
Oct 26 13:50:17 ctl01 kernel: [    0.134290] ACPI: bus type PCI registered
Oct 26 13:50:17 ctl01 kernel: [    0.135320] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
Oct 26 13:50:17 ctl01 kernel: [    0.136151] PCI: Using configuration type 1 for base access
Oct 26 13:50:17 ctl01 kernel: [    0.137554] core: PMU erratum BJ122, BV98, HSD29 workaround disabled, HT off
Oct 26 13:50:17 ctl01 kernel: [    0.140796] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
Oct 26 13:50:17 ctl01 kernel: [    0.141570] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
Oct 26 13:50:17 ctl01 kernel: [    0.144529] ACPI: Added _OSI(Module Device)
Oct 26 13:50:17 ctl01 kernel: [    0.145159] ACPI: Added _OSI(Processor Device)
Oct 26 13:50:17 ctl01 kernel: [    0.146236] ACPI: Added _OSI(3.0 _SCP Extensions)
Oct 26 13:50:17 ctl01 kernel: [    0.148006] ACPI: Added _OSI(Processor Aggregator Device)
Oct 26 13:50:17 ctl01 kernel: [    0.149277] ACPI: Added _OSI(Linux-Dell-Video)
Oct 26 13:50:17 ctl01 kernel: [    0.150348] ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio)
Oct 26 13:50:17 ctl01 kernel: [    0.151601] ACPI: Added _OSI(Linux-HPI-Hybrid-Graphics)
Oct 26 13:50:17 ctl01 kernel: [    0.153674] ACPI: Interpreter enabled
Oct 26 13:50:17 ctl01 kernel: [    0.154620] ACPI: (supports S0 S5)
Oct 26 13:50:17 ctl01 kernel: [    0.155484] ACPI: Using IOAPIC for interrupt routing
Oct 26 13:50:17 ctl01 kernel: [    0.156020] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
Oct 26 13:50:17 ctl01 kernel: [    0.158713] ACPI: Enabled 16 GPEs in block 00 to 0F
Oct 26 13:50:17 ctl01 kernel: [    0.163649] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
Oct 26 13:50:17 ctl01 kernel: [    0.164011] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI]
Oct 26 13:50:17 ctl01 cloud-init[1053]: Cloud-init v. 19.2-36-g059d049c-0ubuntu2~16.04.1 running 'init' at Sat, 26 Oct 2019 13:50:15 +0000. Up 8.53 seconds.
Oct 26 13:50:17 ctl01 cloud-init[1053]: ci-info: ++++++++++++++++++++++++++++++++++++++Net device info+++++++++++++++++++++++++++++++++++++++
Oct 26 13:50:17 ctl01 kernel: [    0.165588] acpi PNP0A03:00: _OSC failed (AE_NOT_FOUND); disabling ASPM
Oct 26 13:50:17 ctl01 cloud-init[1053]: ci-info: +--------+-------+----------------------------+---------------+--------+-------------------+
Oct 26 13:50:17 ctl01 kernel: [    0.167111] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge.
Oct 26 13:50:17 ctl01 kernel: [    0.168359] acpiphp: Slot [3] registered
Oct 26 13:50:17 ctl01 kernel: [    0.169385] acpiphp: Slot [4] registered
Oct 26 13:50:17 ctl01 kernel: [    0.170410] acpiphp: Slot [5] registered
Oct 26 13:50:17 ctl01 cloud-init[1053]: ci-info: | Device |   Up  |          Address           |      Mask     | Scope  |     Hw-Address    |
Oct 26 13:50:17 ctl01 kernel: [    0.171430] acpiphp: Slot [6] registered
Oct 26 13:50:17 ctl01 kernel: [    0.172052] acpiphp: Slot [7] registered
Oct 26 13:50:17 ctl01 kernel: [    0.173093] acpiphp: Slot [9] registered
Oct 26 13:50:17 ctl01 cloud-init[1053]: ci-info: +--------+-------+----------------------------+---------------+--------+-------------------+
Oct 26 13:50:17 ctl01 kernel: [    0.174113] acpiphp: Slot [10] registered
Oct 26 13:50:17 ctl01 cloud-init[1053]: ci-info: |  ens3  |  True |       192.168.11.155       | 255.255.255.0 | global | 52:54:00:d1:44:f2 |
Oct 26 13:50:17 ctl01 kernel: [    0.175148] acpiphp: Slot [11] registered
Oct 26 13:50:17 ctl01 kernel: [    0.176051] acpiphp: Slot [12] registered
Oct 26 13:50:17 ctl01 kernel: [    0.177083] acpiphp: Slot [13] registered
Oct 26 13:50:17 ctl01 kernel: [    0.178121] acpiphp: Slot [14] registered
Oct 26 13:50:17 ctl01 cloud-init[1053]: ci-info: |  ens3  |  True | fe80::5054:ff:fed1:44f2/64 |       .       |  link  | 52:54:00:d1:44:f2 |
Oct 26 13:50:17 ctl01 kernel: [    0.179160] acpiphp: Slot [15] registered
Oct 26 13:50:17 ctl01 kernel: [    0.180051] acpiphp: Slot [16] registered
Oct 26 13:50:17 ctl01 kernel: [    0.181082] acpiphp: Slot [17] registered
Oct 26 13:50:17 ctl01 cloud-init[1053]: ci-info: |  ens4  | False |             .              |       .       |   .    | 52:54:00:cc:bf:d7 |
Oct 26 13:50:17 ctl01 cloud-init[1053]: ci-info: |  ens5  | False |             .              |       .       |   .    | 52:54:00:dd:4a:72 |
Oct 26 13:50:17 ctl01 kernel: [    0.182121] acpiphp: Slot [18] registered
Oct 26 13:50:17 ctl01 cloud-init[1053]: ci-info: |  ens6  | False |             .              |       .       |   .    | 52:54:00:23:fb:5b |
Oct 26 13:50:17 ctl01 kernel: [    0.183160] acpiphp: Slot [19] registered
Oct 26 13:50:17 ctl01 kernel: [    0.184050] acpiphp: Slot [20] registered
Oct 26 13:50:17 ctl01 kernel: [    0.185082] acpiphp: Slot [21] registered
Oct 26 13:50:17 ctl01 kernel: [    0.186117] acpiphp: Slot [22] registered
Oct 26 13:50:17 ctl01 kernel: [    0.187156] acpiphp: Slot [23] registered
Oct 26 13:50:17 ctl01 kernel: [    0.188050] acpiphp: Slot [24] registered
Oct 26 13:50:17 ctl01 kernel: [    0.189085] acpiphp: Slot [25] registered
Oct 26 13:50:17 ctl01 kernel: [    0.190132] acpiphp: Slot [26] registered
Oct 26 13:50:17 ctl01 kernel: [    0.191170] acpiphp: Slot [27] registered
Oct 26 13:50:17 ctl01 kernel: [    0.192051] acpiphp: Slot [28] registered
Oct 26 13:50:17 ctl01 kernel: [    0.193089] acpiphp: Slot [29] registered
Oct 26 13:50:17 ctl01 kernel: [    0.194127] acpiphp: Slot [30] registered
Oct 26 13:50:17 ctl01 kernel: [    0.195163] acpiphp: Slot [31] registered
Oct 26 13:50:17 ctl01 kernel: [    0.196021] PCI host bridge to bus 0000:00
Oct 26 13:50:17 ctl01 kernel: [    0.197031] pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7 window]
Oct 26 13:50:17 ctl01 kernel: [    0.198586] pci_bus 0000:00: root bus resource [io  0x0d00-0xffff window]
Oct 26 13:50:17 ctl01 kernel: [    0.200005] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]
Oct 26 13:50:17 ctl01 kernel: [    0.201763] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]
Oct 26 13:50:17 ctl01 kernel: [    0.203516] pci_bus 0000:00: root bus resource [bus 00-ff]
Oct 26 13:50:17 ctl01 kernel: [    0.204055] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000
Oct 26 13:50:17 ctl01 kernel: [    0.204668] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100
Oct 26 13:50:17 ctl01 kernel: [    0.205454] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180
Oct 26 13:50:17 ctl01 kernel: [    0.212985] pci 0000:00:01.1: reg 0x20: [io  0xc140-0xc14f]
Oct 26 13:50:17 ctl01 kernel: [    0.216707] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io  0x01f0-0x01f7]
Oct 26 13:50:17 ctl01 kernel: [    0.218755] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io  0x03f6]
Oct 26 13:50:17 ctl01 kernel: [    0.220005] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io  0x0170-0x0177]
Oct 26 13:50:17 ctl01 kernel: [    0.221627] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io  0x0376]
Oct 26 13:50:17 ctl01 kernel: [    0.223358] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000
Oct 26 13:50:17 ctl01 kernel: [    0.223963] pci 0000:00:01.3: quirk: [io  0x0600-0x063f] claimed by PIIX4 ACPI
Oct 26 13:50:17 ctl01 kernel: [    0.224017] pci 0000:00:01.3: quirk: [io  0x0700-0x070f] claimed by PIIX4 SMB
Oct 26 13:50:17 ctl01 kernel: [    0.226044] pci 0000:00:02.0: [1013:00b8] type 00 class 0x030000
Oct 26 13:50:17 ctl01 kernel: [    0.228011] pci 0000:00:02.0: reg 0x10: [mem 0xfc000000-0xfdffffff pref]
Oct 26 13:50:17 ctl01 kernel: [    0.230713] pci 0000:00:02.0: reg 0x14: [mem 0xfebd0000-0xfebd0fff]
Oct 26 13:50:17 ctl01 kernel: [    0.243728] pci 0000:00:02.0: reg 0x30: [mem 0xfebc0000-0xfebcffff pref]
Oct 26 13:50:17 ctl01 kernel: [    0.244265] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000
Oct 26 13:50:17 ctl01 kernel: [    0.246564] pci 0000:00:03.0: reg 0x10: [io  0xc040-0xc05f]
Oct 26 13:50:17 ctl01 kernel: [    0.248006] pci 0000:00:03.0: reg 0x14: [mem 0xfebd1000-0xfebd1fff]
Oct 26 13:50:17 ctl01 kernel: [    0.259010] pci 0000:00:03.0: reg 0x30: [mem 0xfeac0000-0xfeafffff pref]
Oct 26 13:50:17 ctl01 kernel: [    0.259461] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000
Oct 26 13:50:17 ctl01 kernel: [    0.261131] pci 0000:00:04.0: reg 0x10: [io  0xc060-0xc07f]
Oct 26 13:50:17 ctl01 kernel: [    0.263131] pci 0000:00:04.0: reg 0x14: [mem 0xfebd2000-0xfebd2fff]
Oct 26 13:50:17 ctl01 kernel: [    0.274089] pci 0000:00:04.0: reg 0x30: [mem 0xfeb00000-0xfeb3ffff pref]
Oct 26 13:50:17 ctl01 kernel: [    0.274546] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000
Oct 26 13:50:17 ctl01 kernel: [    0.277045] pci 0000:00:05.0: reg 0x10: [io  0xc080-0xc09f]
Oct 26 13:50:17 ctl01 kernel: [    0.279014] pci 0000:00:05.0: reg 0x14: [mem 0xfebd3000-0xfebd3fff]
Oct 26 13:50:17 ctl01 kernel: [    0.289967] pci 0000:00:05.0: reg 0x30: [mem 0xfeb40000-0xfeb7ffff pref]
Oct 26 13:50:17 ctl01 kernel: [    0.290414] pci 0000:00:06.0: [1af4:1000] type 00 class 0x020000
Oct 26 13:50:17 ctl01 kernel: [    0.292006] pci 0000:00:06.0: reg 0x10: [io  0xc0a0-0xc0bf]
Oct 26 13:50:17 ctl01 kernel: [    0.294004] pci 0000:00:06.0: reg 0x14: [mem 0xfebd4000-0xfebd4fff]
Oct 26 13:50:17 ctl01 kernel: [    0.305989] pci 0000:00:06.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref]
Oct 26 13:50:17 ctl01 kernel: [    0.306437] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000
Oct 26 13:50:17 ctl01 kernel: [    0.308006] pci 0000:00:07.0: reg 0x10: [io  0xc000-0xc03f]
Oct 26 13:50:17 ctl01 kernel: [    0.312008] pci 0000:00:07.0: reg 0x14: [mem 0xfebd5000-0xfebd5fff]
Oct 26 13:50:17 ctl01 kernel: [    0.323835] pci 0000:00:08.0: [8086:2934] type 00 class 0x0c0300
Oct 26 13:50:17 ctl01 kernel: [    0.328548] pci 0000:00:08.0: reg 0x20: [io  0xc0c0-0xc0df]
Oct 26 13:50:17 ctl01 kernel: [    0.331490] pci 0000:00:08.1: [8086:2935] type 00 class 0x0c0300
Oct 26 13:50:17 ctl01 kernel: [    0.338256] pci 0000:00:08.1: reg 0x20: [io  0xc0e0-0xc0ff]
Oct 26 13:50:17 ctl01 kernel: [    0.340828] pci 0000:00:08.2: [8086:2936] type 00 class 0x0c0300
Oct 26 13:50:17 ctl01 kernel: [    0.347475] pci 0000:00:08.2: reg 0x20: [io  0xc100-0xc11f]
Oct 26 13:50:17 ctl01 kernel: [    0.350073] pci 0000:00:08.7: [8086:293a] type 00 class 0x0c0320
Oct 26 13:50:17 ctl01 kernel: [    0.351279] pci 0000:00:08.7: reg 0x10: [mem 0xfebd6000-0xfebd6fff]
Oct 26 13:50:17 ctl01 kernel: [    0.359579] pci 0000:00:09.0: [1af4:1002] type 00 class 0x00ff00
Oct 26 13:50:17 ctl01 kernel: [    0.360588] pci 0000:00:09.0: reg 0x10: [io  0xc120-0xc13f]
Oct 26 13:50:17 ctl01 kernel: [    0.369597] ACPI: PCI Interrupt Link [LNKA] (IRQs 5 *10 11)
Oct 26 13:50:17 ctl01 kernel: [    0.371226] ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11)
Oct 26 13:50:17 ctl01 kernel: [    0.372157] ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11)
Oct 26 13:50:17 ctl01 kernel: [    0.373732] ACPI: PCI Interrupt Link [LNKD] (IRQs 5 10 *11)
Oct 26 13:50:17 ctl01 kernel: [    0.375222] ACPI: PCI Interrupt Link [LNKS] (IRQs *9)
Oct 26 13:50:17 ctl01 kernel: [    0.377110] SCSI subsystem initialized
Oct 26 13:50:17 ctl01 kernel: [    0.378198] libata version 3.00 loaded.
Oct 26 13:50:17 ctl01 kernel: [    0.378198] pci 0000:00:02.0: vgaarb: setting as boot VGA device
Oct 26 13:50:17 ctl01 kernel: [    0.378198] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none
Oct 26 13:50:17 ctl01 kernel: [    0.380015] pci 0000:00:02.0: vgaarb: bridge control possible
Oct 26 13:50:17 ctl01 kernel: [    0.381462] vgaarb: loaded
Oct 26 13:50:17 ctl01 kernel: [    0.382263] ACPI: bus type USB registered
Oct 26 13:50:17 ctl01 kernel: [    0.383354] usbcore: registered new interface driver usbfs
Oct 26 13:50:17 ctl01 kernel: [    0.384014] usbcore: registered new interface driver hub
Oct 26 13:50:17 ctl01 kernel: [    0.385419] usbcore: registered new device driver usb
Oct 26 13:50:17 ctl01 kernel: [    0.386797] EDAC MC: Ver: 3.0.0
Oct 26 13:50:17 ctl01 kernel: [    0.388167] PCI: Using ACPI for IRQ routing
Oct 26 13:50:17 ctl01 kernel: [    0.389157] PCI: pci_cache_line_size set to 64 bytes
Oct 26 13:50:17 ctl01 kernel: [    0.389510] e820: reserve RAM buffer [mem 0x0009fc00-0x0009ffff]
Oct 26 13:50:17 ctl01 kernel: [    0.389512] e820: reserve RAM buffer [mem 0xbffdf000-0xbfffffff]
Oct 26 13:50:17 ctl01 kernel: [    0.389620] NetLabel: Initializing
Oct 26 13:50:17 ctl01 kernel: [    0.390563] NetLabel:  domain hash size = 128
Oct 26 13:50:17 ctl01 kernel: [    0.392005] NetLabel:  protocols = UNLABELED CIPSOv4 CALIPSO
Oct 26 13:50:17 ctl01 kernel: [    0.393454] NetLabel:  unlabeled traffic allowed by default
Oct 26 13:50:17 ctl01 kernel: [    0.394888] clocksource: Switched to clocksource kvm-clock
Oct 26 13:50:17 ctl01 kernel: [    0.405760] VFS: Disk quotas dquot_6.6.0
Oct 26 13:50:17 ctl01 kernel: [    0.406837] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
Oct 26 13:50:17 ctl01 kernel: [    0.408640] AppArmor: AppArmor Filesystem Enabled
Oct 26 13:50:17 ctl01 kernel: [    0.409890] pnp: PnP ACPI init
Oct 26 13:50:17 ctl01 kernel: [    0.410813] pnp 00:00: Plug and Play ACPI device, IDs PNP0b00 (active)
Oct 26 13:50:17 ctl01 kernel: [    0.410859] pnp 00:01: Plug and Play ACPI device, IDs PNP0303 (active)
Oct 26 13:50:17 ctl01 kernel: [    0.410897] pnp 00:02: Plug and Play ACPI device, IDs PNP0f13 (active)
Oct 26 13:50:17 ctl01 kernel: [    0.410933] pnp 00:03: [dma 2]
Oct 26 13:50:17 ctl01 kernel: [    0.410947] pnp 00:03: Plug and Play ACPI device, IDs PNP0700 (active)
Oct 26 13:50:17 ctl01 kernel: [    0.411053] pnp 00:04: Plug and Play ACPI device, IDs PNP0501 (active)
Oct 26 13:50:17 ctl01 kernel: [    0.411367] pnp: PnP ACPI: found 5 devices
Oct 26 13:50:17 ctl01 kernel: [    0.419256] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
Oct 26 13:50:17 ctl01 kernel: [    0.421463] pci_bus 0000:00: resource 4 [io  0x0000-0x0cf7 window]
Oct 26 13:50:17 ctl01 kernel: [    0.421464] pci_bus 0000:00: resource 5 [io  0x0d00-0xffff window]
Oct 26 13:50:17 ctl01 kernel: [    0.421466] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]
Oct 26 13:50:17 ctl01 kernel: [    0.421467] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window]
Oct 26 13:50:17 ctl01 kernel: [    0.421538] NET: Registered protocol family 2
Oct 26 13:50:17 ctl01 kernel: [    0.423436] TCP established hash table entries: 131072 (order: 8, 1048576 bytes)
Oct 26 13:50:17 ctl01 kernel: [    0.425509] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
Oct 26 13:50:17 ctl01 kernel: [    0.427249] TCP: Hash tables configured (established 131072 bind 65536)
Oct 26 13:50:17 ctl01 kernel: [    0.429447] UDP hash table entries: 8192 (order: 6, 262144 bytes)
Oct 26 13:50:17 ctl01 kernel: [    0.431014] UDP-Lite hash table entries: 8192 (order: 6, 262144 bytes)
Oct 26 13:50:17 ctl01 kernel: [    0.432717] NET: Registered protocol family 1
Oct 26 13:50:17 ctl01 kernel: [    0.433892] pci 0000:00:00.0: Limiting direct PCI/PCI transfers
Oct 26 13:50:17 ctl01 kernel: [    0.435381] pci 0000:00:01.0: PIIX3: Enabling Passive Release
Oct 26 13:50:17 ctl01 kernel: [    0.436856] pci 0000:00:01.0: Activating ISA DMA hang workarounds
Oct 26 13:50:17 ctl01 kernel: [    0.438451] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]
Oct 26 13:50:17 ctl01 kernel: [    0.471819] ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 11
Oct 26 13:50:17 ctl01 kernel: [    0.535367] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 10
Oct 26 13:50:17 ctl01 kernel: [    0.596915] ACPI: PCI Interrupt Link [LNKB] enabled at IRQ 10
Oct 26 13:50:17 ctl01 kernel: [    0.655651] ACPI: PCI Interrupt Link [LNKC] enabled at IRQ 11
Oct 26 13:50:17 ctl01 kernel: [    0.685861] PCI: CLS 0 bytes, default 64
Oct 26 13:50:17 ctl01 kernel: [    0.685899] Unpacking initramfs...
Oct 26 13:50:17 ctl01 kernel: [    1.414908] Freeing initrd memory: 53652K
Oct 26 13:50:17 ctl01 kernel: [    1.416028] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
Oct 26 13:50:17 ctl01 kernel: [    1.417522] software IO TLB: mapped [mem 0xbbfdf000-0xbffdf000] (64MB)
Oct 26 13:50:17 ctl01 kernel: [    1.419064] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x285c3aeaff3, max_idle_ns: 440795255742 ns
Oct 26 13:50:17 ctl01 kernel: [    1.421404] Scanning for low memory corruption every 60 seconds
Oct 26 13:50:17 ctl01 kernel: [    1.423643] Initialise system trusted keyrings
Oct 26 13:50:17 ctl01 kernel: [    1.424749] Key type blacklist registered
Oct 26 13:50:17 ctl01 kernel: [    1.425898] workingset: timestamp_bits=36 max_order=22 bucket_order=0
Oct 26 13:50:17 ctl01 kernel: [    1.428892] zbud: loaded
Oct 26 13:50:17 ctl01 kernel: [    1.430198] squashfs: version 4.0 (2009/01/31) Phillip Lougher
Oct 26 13:50:17 ctl01 kernel: [    1.431819] fuse init (API version 7.26)
Oct 26 13:50:17 ctl01 kernel: [    1.434450] Key type asymmetric registered
Oct 26 13:50:17 ctl01 kernel: [    1.435469] Asymmetric key parser 'x509' registered
Oct 26 13:50:17 ctl01 kernel: [    1.436668] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 246)
Oct 26 13:50:17 ctl01 kernel: [    1.438511] io scheduler noop registered
Oct 26 13:50:17 ctl01 kernel: [    1.439491] io scheduler deadline registered
Oct 26 13:50:17 ctl01 kernel: [    1.440572] io scheduler cfq registered (default)
Oct 26 13:50:17 ctl01 kernel: [    1.442192] intel_idle: Please enable MWAIT in BIOS SETUP
Oct 26 13:50:17 ctl01 kernel: [    1.442266] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
Oct 26 13:50:17 ctl01 kernel: [    1.444440] ACPI: Power Button [PWRF]
Oct 26 13:50:17 ctl01 kernel: [    1.474326] virtio-pci 0000:00:03.0: virtio_pci: leaving for legacy driver
Oct 26 13:50:17 ctl01 kernel: [    1.505060] virtio-pci 0000:00:04.0: virtio_pci: leaving for legacy driver
Oct 26 13:50:17 ctl01 kernel: [    1.535727] virtio-pci 0000:00:05.0: virtio_pci: leaving for legacy driver
Oct 26 13:50:17 ctl01 kernel: [    1.566413] virtio-pci 0000:00:06.0: virtio_pci: leaving for legacy driver
Oct 26 13:50:17 ctl01 kernel: [    1.597610] virtio-pci 0000:00:07.0: virtio_pci: leaving for legacy driver
Oct 26 13:50:17 ctl01 kernel: [    1.627978] virtio-pci 0000:00:09.0: virtio_pci: leaving for legacy driver
Oct 26 13:50:17 ctl01 kernel: [    1.630689] Serial: 8250/16550 driver, 32 ports, IRQ sharing enabled
Oct 26 13:50:17 ctl01 kernel: [    1.657471] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
Oct 26 13:50:17 ctl01 kernel: [    1.661007] Linux agpgart interface v0.103
Oct 26 13:50:17 ctl01 kernel: [    1.664850] loop: module loaded
Oct 26 13:50:17 ctl01 kernel: [    1.665767] ata_piix 0000:00:01.1: version 2.13
Oct 26 13:50:17 ctl01 kernel: [    1.667005] scsi host0: ata_piix
Oct 26 13:50:17 ctl01 kernel: [    1.668020] scsi host1: ata_piix
Oct 26 13:50:17 ctl01 kernel: [    1.668889] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc140 irq 14
Oct 26 13:50:17 ctl01 kernel: [    1.670452] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc148 irq 15
Oct 26 13:50:17 ctl01 kernel: [    1.672082] libphy: Fixed MDIO Bus: probed
Oct 26 13:50:17 ctl01 kernel: [    1.673214] tun: Universal TUN/TAP device driver, 1.6
Oct 26 13:50:17 ctl01 kernel: [    1.674488] PPP generic driver version 2.4.2
Oct 26 13:50:17 ctl01 kernel: [    1.675615] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
Oct 26 13:50:17 ctl01 kernel: [    1.677234] ehci-pci: EHCI PCI platform driver
Oct 26 13:50:17 ctl01 kernel: [    1.707939] ehci-pci 0000:00:08.7: EHCI Host Controller
Oct 26 13:50:17 ctl01 kernel: [    1.717539] ehci-pci 0000:00:08.7: new USB bus registered, assigned bus number 1
Oct 26 13:50:17 ctl01 kernel: [    1.719516] ehci-pci 0000:00:08.7: irq 11, io mem 0xfebd6000
Oct 26 13:50:17 ctl01 kernel: [    1.736055] ehci-pci 0000:00:08.7: USB 2.0 started, EHCI 1.00
Oct 26 13:50:17 ctl01 kernel: [    1.738012] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
Oct 26 13:50:17 ctl01 kernel: [    1.740210] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
Oct 26 13:50:17 ctl01 kernel: [    1.742594] usb usb1: Product: EHCI Host Controller
Oct 26 13:50:17 ctl01 kernel: [    1.744223] usb usb1: Manufacturer: Linux 4.15.0-66-generic ehci_hcd
Oct 26 13:50:17 ctl01 kernel: [    1.746277] usb usb1: SerialNumber: 0000:00:08.7
Oct 26 13:50:17 ctl01 kernel: [    1.748035] hub 1-0:1.0: USB hub found
Oct 26 13:50:17 ctl01 kernel: [    1.749005] hub 1-0:1.0: 6 ports detected
Oct 26 13:50:17 ctl01 kernel: [    1.750192] ehci-platform: EHCI generic platform driver
Oct 26 13:50:17 ctl01 kernel: [    1.751441] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
Oct 26 13:50:17 ctl01 kernel: [    1.752878] ohci-pci: OHCI PCI platform driver
Oct 26 13:50:17 ctl01 kernel: [    1.753970] ohci-platform: OHCI generic platform driver
Oct 26 13:50:17 ctl01 kernel: [    1.755215] uhci_hcd: USB Universal Host Controller Interface driver
Oct 26 13:50:17 ctl01 kernel: [    1.785923] uhci_hcd 0000:00:08.0: UHCI Host Controller
Oct 26 13:50:17 ctl01 kernel: [    1.787173] uhci_hcd 0000:00:08.0: new USB bus registered, assigned bus number 2
Oct 26 13:50:17 ctl01 kernel: [    1.788943] uhci_hcd 0000:00:08.0: detected 2 ports
Oct 26 13:50:17 ctl01 kernel: [    1.790185] uhci_hcd 0000:00:08.0: irq 11, io base 0x0000c0c0
Oct 26 13:50:17 ctl01 kernel: [    1.791600] usb usb2: New USB device found, idVendor=1d6b, idProduct=0001
Oct 26 13:50:17 ctl01 kernel: [    1.793242] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
Oct 26 13:50:17 ctl01 kernel: [    1.794957] usb usb2: Product: UHCI Host Controller
Oct 26 13:50:17 ctl01 kernel: [    1.796271] usb usb2: Manufacturer: Linux 4.15.0-66-generic uhci_hcd
Oct 26 13:50:17 ctl01 kernel: [    1.797815] usb usb2: SerialNumber: 0000:00:08.0
Oct 26 13:50:17 ctl01 kernel: [    1.799071] hub 2-0:1.0: USB hub found
Oct 26 13:50:17 ctl01 kernel: [    1.800020] hub 2-0:1.0: 2 ports detected
Oct 26 13:50:17 ctl01 kernel: [    1.832393] uhci_hcd 0000:00:08.1: UHCI Host Controller
Oct 26 13:50:17 ctl01 kernel: [    1.833499] ata1.01: NODEV after polling detection
Oct 26 13:50:17 ctl01 kernel: [    1.833980] uhci_hcd 0000:00:08.1: new USB bus registered, assigned bus number 3
Oct 26 13:50:17 ctl01 kernel: [    1.834446] ata1.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100
Oct 26 13:50:17 ctl01 kernel: [    1.836015] uhci_hcd 0000:00:08.1: detected 2 ports
Oct 26 13:50:17 ctl01 kernel: [    1.838329] ata1.00: configured for MWDMA2
Oct 26 13:50:17 ctl01 kernel: [    1.839075] uhci_hcd 0000:00:08.1: irq 10, io base 0x0000c0e0
Oct 26 13:50:17 ctl01 kernel: [    1.841365] scsi 0:0:0:0: CD-ROM            QEMU     QEMU DVD-ROM     2.5+ PQ: 0 ANSI: 5
Oct 26 13:50:17 ctl01 kernel: [    1.842190] usb usb3: New USB device found, idVendor=1d6b, idProduct=0001
Oct 26 13:50:17 ctl01 kernel: [    1.846067] sr 0:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray
Oct 26 13:50:17 ctl01 kernel: [    1.846355] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
Oct 26 13:50:17 ctl01 kernel: [    1.848128] cdrom: Uniform CD-ROM driver Revision: 3.20
Oct 26 13:50:17 ctl01 kernel: [    1.850088] usb usb3: Product: UHCI Host Controller
Oct 26 13:50:17 ctl01 kernel: [    1.851705] sr 0:0:0:0: Attached scsi CD-ROM sr0
Oct 26 13:50:17 ctl01 kernel: [    1.852905] usb usb3: Manufacturer: Linux 4.15.0-66-generic uhci_hcd
Oct 26 13:50:17 ctl01 kernel: [    1.852998] sr 0:0:0:0: Attached scsi generic sg0 type 5
Oct 26 13:50:17 ctl01 kernel: [    1.854453] usb usb3: SerialNumber: 0000:00:08.1
Oct 26 13:50:17 ctl01 kernel: [    1.857051] hub 3-0:1.0: USB hub found
Oct 26 13:50:17 ctl01 kernel: [    1.858036] hub 3-0:1.0: 2 ports detected
Oct 26 13:50:17 ctl01 kernel: [    1.888643] uhci_hcd 0000:00:08.2: UHCI Host Controller
Oct 26 13:50:17 ctl01 kernel: [    1.889921] uhci_hcd 0000:00:08.2: new USB bus registered, assigned bus number 4
Oct 26 13:50:17 ctl01 kernel: [    1.891709] uhci_hcd 0000:00:08.2: detected 2 ports
Oct 26 13:50:17 ctl01 kernel: [    1.892979] uhci_hcd 0000:00:08.2: irq 10, io base 0x0000c100
Oct 26 13:50:17 ctl01 kernel: [    1.894417] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001
Oct 26 13:50:17 ctl01 kernel: [    1.896050] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
Oct 26 13:50:17 ctl01 kernel: [    1.897952] usb usb4: Product: UHCI Host Controller
Oct 26 13:50:17 ctl01 kernel: [    1.899160] usb usb4: Manufacturer: Linux 4.15.0-66-generic uhci_hcd
Oct 26 13:50:17 ctl01 kernel: [    1.900694] usb usb4: SerialNumber: 0000:00:08.2
Oct 26 13:50:17 ctl01 kernel: [    1.902058] hub 4-0:1.0: USB hub found
Oct 26 13:50:17 ctl01 kernel: [    1.903054] hub 4-0:1.0: 2 ports detected
Oct 26 13:50:17 ctl01 kernel: [    1.904283] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12
Oct 26 13:50:17 ctl01 kernel: [    1.907248] serio: i8042 KBD port at 0x60,0x64 irq 1
Oct 26 13:50:17 ctl01 kernel: [    1.908530] serio: i8042 AUX port at 0x60,0x64 irq 12
Oct 26 13:50:17 ctl01 kernel: [    1.909959] mousedev: PS/2 mouse device common for all mice
Oct 26 13:50:17 ctl01 kernel: [    1.911570] rtc_cmos 00:00: RTC can wake from S4
Oct 26 13:50:17 ctl01 kernel: [    1.913284] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1
Oct 26 13:50:17 ctl01 kernel: [    1.915406] rtc_cmos 00:00: rtc core: registered rtc_cmos as rtc0
Oct 26 13:50:17 ctl01 kernel: [    1.917115] rtc_cmos 00:00: alarms up to one day, 114 bytes nvram
Oct 26 13:50:17 ctl01 kernel: [    1.918717] i2c /dev entries driver
Oct 26 13:50:17 ctl01 kernel: [    1.919664] device-mapper: uevent: version 1.0.3
Oct 26 13:50:17 ctl01 kernel: [    1.920990] device-mapper: ioctl: 4.37.0-ioctl (2017-09-20) initialised: dm-devel@redhat.com
Oct 26 13:50:17 ctl01 kernel: [    1.923411] ledtrig-cpu: registered to indicate activity on CPUs
Oct 26 13:50:17 ctl01 kernel: [    1.925429] NET: Registered protocol family 10
Oct 26 13:50:17 ctl01 kernel: [    1.930801] Segment Routing with IPv6
Oct 26 13:50:17 ctl01 kernel: [    1.931838] NET: Registered protocol family 17
Oct 26 13:50:17 ctl01 kernel: [    1.933089] Key type dns_resolver registered
Oct 26 13:50:17 ctl01 kernel: [    1.935016] mce: Using 10 MCE banks
Oct 26 13:50:17 ctl01 kernel: [    1.935956] RAS: Correctable Errors collector initialized.
Oct 26 13:50:17 ctl01 kernel: [    1.937363] sched_clock: Marking stable (1937347428, 0)->(2511714585, -574367157)
Oct 26 13:50:17 ctl01 kernel: [    1.939669] registered taskstats version 1
Oct 26 13:50:17 ctl01 kernel: [    1.940787] Loading compiled-in X.509 certificates
Oct 26 13:50:17 ctl01 kernel: [    1.945584] Loaded X.509 cert 'Build time autogenerated kernel key: b403aa2f6b5a8b809b81566991891c544ad7d3fe'
Oct 26 13:50:17 ctl01 kernel: [    1.948492] zswap: loaded using pool lzo/zbud
Oct 26 13:50:17 ctl01 kernel: [    1.954090] Key type big_key registered
Oct 26 13:50:17 ctl01 kernel: [    1.955152] Key type trusted registered
Oct 26 13:50:17 ctl01 kernel: [    1.959995] Key type encrypted registered
Oct 26 13:50:17 ctl01 kernel: [    1.961159] AppArmor: AppArmor sha1 policy hashing enabled
Oct 26 13:50:17 ctl01 kernel: [    1.962661] ima: No TPM chip found, activating TPM-bypass! (rc=-19)
Oct 26 13:50:17 ctl01 kernel: [    1.964277] ima: Allocated hash algorithm: sha1
Oct 26 13:50:17 ctl01 kernel: [    1.965534] evm: HMAC attrs: 0x1
Oct 26 13:50:17 ctl01 kernel: [    1.966933]   Magic number: 7:146:838
Oct 26 13:50:17 ctl01 kernel: [    1.968131] rtc_cmos 00:00: setting system clock to 2019-10-26 13:50:08 UTC (1572097808)
Oct 26 13:50:17 ctl01 kernel: [    1.970256] BIOS EDD facility v0.16 2004-Jun-25, 0 devices found
Oct 26 13:50:17 ctl01 kernel: [    1.971704] EDD information not available.
Oct 26 13:50:17 ctl01 kernel: [    2.341470] Freeing unused kernel image memory: 2428K
Oct 26 13:50:17 ctl01 kernel: [    2.356077] Write protecting the kernel read-only data: 20480k
Oct 26 13:50:17 ctl01 kernel: [    2.359286] Freeing unused kernel image memory: 2008K
Oct 26 13:50:17 ctl01 kernel: [    2.361489] Freeing unused kernel image memory: 1936K
Oct 26 13:50:17 ctl01 kernel: [    2.373531] x86/mm: Checked W+X mappings: passed, no W+X pages found.
Oct 26 13:50:17 ctl01 kernel: [    2.375029] x86/mm: Checking user space page tables
Oct 26 13:50:17 ctl01 kernel: [    2.383418] x86/mm: Checked W+X mappings: passed, no W+X pages found.
Oct 26 13:50:17 ctl01 kernel: [    2.403434] random: systemd-udevd: uninitialized urandom read (16 bytes read)
Oct 26 13:50:17 ctl01 kernel: [    2.404270] random: udevadm: uninitialized urandom read (16 bytes read)
Oct 26 13:50:17 ctl01 kernel: [    2.405422] random: systemd-udevd: uninitialized urandom read (16 bytes read)
Oct 26 13:50:17 ctl01 kernel: [    2.467008]  vda: vda1
Oct 26 13:50:17 ctl01 kernel: [    2.470010] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4
Oct 26 13:50:17 ctl01 kernel: [    2.473219] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3
Oct 26 13:50:17 ctl01 kernel: [    2.477534] FDC 0 is a S82078B
Oct 26 13:50:17 ctl01 kernel: [    2.481942] AVX version of gcm_enc/dec engaged.
Oct 26 13:50:17 ctl01 kernel: [    2.483065] AES CTR mode by8 optimization enabled
Oct 26 13:50:17 ctl01 kernel: [    2.491031] virtio_net virtio0 ens3: renamed from eth0
Oct 26 13:50:17 ctl01 kernel: [    2.494331] [TTM] Zone  kernel: Available graphics memory: 7180184 kiB
Oct 26 13:50:17 ctl01 kernel: [    2.497357] [TTM] Zone   dma32: Available graphics memory: 2097152 kiB
Oct 26 13:50:17 ctl01 kernel: [    2.498913] [TTM] Initializing pool allocator
Oct 26 13:50:17 ctl01 kernel: [    2.500022] [TTM] Initializing DMA pool allocator
Oct 26 13:50:17 ctl01 kernel: [    2.502192] [drm] fb mappable at 0xFC000000
Oct 26 13:50:17 ctl01 kernel: [    2.503870] [drm] vram aper at 0xFC000000
Oct 26 13:50:17 ctl01 kernel: [    2.505059] [drm] size 33554432
Oct 26 13:50:17 ctl01 kernel: [    2.506076] [drm] fb depth is 24
Oct 26 13:50:17 ctl01 kernel: [    2.507232] [drm]    pitch is 3072
Oct 26 13:50:17 ctl01 kernel: [    2.508319] fbcon: cirrusdrmfb (fb0) is primary device
Oct 26 13:50:17 ctl01 kernel: [    2.525508] virtio_net virtio1 ens4: renamed from eth1
Oct 26 13:50:17 ctl01 kernel: [    2.528910] Console: switching to colour frame buffer device 128x48
Oct 26 13:50:17 ctl01 kernel: [    2.534248] cirrus 0000:00:02.0: fb0: cirrusdrmfb frame buffer device
Oct 26 13:50:17 ctl01 kernel: [    2.552024] [drm] Initialized cirrus 1.0.0 20110418 for 0000:00:02.0 on minor 0
Oct 26 13:50:17 ctl01 kernel: [    2.552137] virtio_net virtio2 ens5: renamed from eth2
Oct 26 13:50:17 ctl01 kernel: [    2.580222] virtio_net virtio3 ens6: renamed from eth3
Oct 26 13:50:17 ctl01 kernel: [    2.756019] raid6: sse2x1   gen()  7309 MB/s
Oct 26 13:50:17 ctl01 kernel: [    2.804010] raid6: sse2x1   xor()  5773 MB/s
Oct 26 13:50:17 ctl01 kernel: [    2.852007] raid6: sse2x2   gen()  9040 MB/s
Oct 26 13:50:17 ctl01 kernel: [    2.900007] raid6: sse2x2   xor()  6514 MB/s
Oct 26 13:50:17 ctl01 kernel: [    2.948011] raid6: sse2x4   gen() 10618 MB/s
Oct 26 13:50:17 ctl01 kernel: [    2.996008] raid6: sse2x4   xor()  7706 MB/s
Oct 26 13:50:17 ctl01 kernel: [    2.996784] raid6: using algorithm sse2x4 gen() 10618 MB/s
Oct 26 13:50:17 ctl01 kernel: [    2.997766] raid6: .... xor() 7706 MB/s, rmw enabled
Oct 26 13:50:17 ctl01 kernel: [    2.998648] raid6: using ssse3x2 recovery algorithm
Oct 26 13:50:17 ctl01 kernel: [    3.001815] xor: automatically using best checksumming function   avx       
Oct 26 13:50:17 ctl01 kernel: [    3.004912] async_tx: api initialized (async)
Oct 26 13:50:17 ctl01 kernel: [    3.068595] Btrfs loaded, crc32c=crc32c-intel
Oct 26 13:50:17 ctl01 kernel: [    3.125556] EXT4-fs (vda1): mounted filesystem with ordered data mode. Opts: (null)
Oct 26 13:50:17 ctl01 kernel: [    3.327229] random: fast init done
Oct 26 13:50:17 ctl01 kernel: [    3.731884] EXT4-fs (vda1): re-mounted. Opts: (null)
Oct 26 13:50:17 ctl01 kernel: [    3.749533] Loading iSCSI transport class v2.0-870.
Oct 26 13:50:17 ctl01 kernel: [    3.768856] iscsi: registered transport (tcp)
Oct 26 13:50:17 ctl01 kernel: [    3.798457] iscsi: registered transport (iser)
Oct 26 13:50:17 ctl01 kernel: [    4.013638] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0
Oct 26 13:50:17 ctl01 kernel: [    4.384515] audit: type=1400 audit(1572097810.912:2): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/bin/lxc-start" pid=772 comm="apparmor_parser"
Oct 26 13:50:17 ctl01 kernel: [    4.448508] EDAC sbridge: Seeking for: PCI ID 8086:0ea0
Oct 26 13:50:17 ctl01 kernel: [    4.448515] EDAC sbridge:  Ver: 1.1.2 
Oct 26 13:50:17 ctl01 kernel: [    4.452155] audit: type=1400 audit(1572097810.980:3): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/lib/lxd/lxd-bridge-proxy" pid=773 comm="apparmor_parser"
Oct 26 13:50:17 ctl01 kernel: [    4.486695] audit: type=1400 audit(1572097811.012:4): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/sbin/tcpdump" pid=777 comm="apparmor_parser"
Oct 26 13:50:17 ctl01 kernel: [    4.586341] audit: type=1400 audit(1572097811.112:5): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/sbin/dhclient" pid=770 comm="apparmor_parser"
Oct 26 13:50:17 ctl01 kernel: [    4.586741] audit: type=1400 audit(1572097811.112:6): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/lib/NetworkManager/nm-dhcp-client.action" pid=770 comm="apparmor_parser"
Oct 26 13:50:17 ctl01 kernel: [    4.587125] audit: type=1400 audit(1572097811.112:7): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/lib/NetworkManager/nm-dhcp-helper" pid=770 comm="apparmor_parser"
Oct 26 13:50:17 ctl01 kernel: [    4.587481] audit: type=1400 audit(1572097811.112:8): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/lib/connman/scripts/dhclient-script" pid=770 comm="apparmor_parser"
Oct 26 13:50:17 ctl01 kernel: [    4.614150] audit: type=1400 audit(1572097811.140:9): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/lib/snapd/snap-confine" pid=775 comm="apparmor_parser"
Oct 26 13:50:17 ctl01 kernel: [    4.614366] audit: type=1400 audit(1572097811.140:10): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/lib/snapd/snap-confine//mount-namespace-capture-helper" pid=775 comm="apparmor_parser"
Oct 26 13:50:17 ctl01 kernel: [    5.204785] ISO 9660 Extensions: Microsoft Joliet Level 3
Oct 26 13:50:17 ctl01 kernel: [    5.210210] ISO 9660 Extensions: RRIP_1991A
Oct 26 13:50:17 ctl01 kernel: [    9.026359] EXT4-fs (vda1): resizing filesystem from 786171 to 26214139 blocks
Oct 26 13:50:17 ctl01 kernel: [    9.176018] EXT4-fs (vda1): resized filesystem to 26214139
Oct 26 13:50:17 ctl01 kernel: [   11.055374] new mount options do not match the existing superblock, will be ignored
Oct 26 13:50:17 ctl01 cloud-init[1053]: ci-info: |   lo   |  True |         127.0.0.1          |   255.0.0.0   |  host  |         .         |
Oct 26 13:50:17 ctl01 cloud-init[1053]: ci-info: |   lo   |  True |          ::1/128           |       .       |  host  |         .         |
Oct 26 13:50:17 ctl01 cloud-init[1053]: ci-info: +--------+-------+----------------------------+---------------+--------+-------------------+
Oct 26 13:50:17 ctl01 cloud-init[1053]: ci-info: ++++++++++++++++++++++++++++++Route IPv4 info++++++++++++++++++++++++++++++
Oct 26 13:50:17 ctl01 cloud-init[1053]: ci-info: +-------+--------------+--------------+---------------+-----------+-------+
Oct 26 13:50:17 ctl01 cloud-init[1053]: ci-info: | Route | Destination  |   Gateway    |    Genmask    | Interface | Flags |
Oct 26 13:50:17 ctl01 cloud-init[1053]: ci-info: +-------+--------------+--------------+---------------+-----------+-------+
Oct 26 13:50:17 ctl01 cloud-init[1053]: ci-info: |   0   |   0.0.0.0    | 192.168.11.3 |    0.0.0.0    |    ens3   |   UG  |
Oct 26 13:50:17 ctl01 cloud-init[1053]: ci-info: |   1   | 192.168.11.0 |   0.0.0.0    | 255.255.255.0 |    ens3   |   U   |
Oct 26 13:50:17 ctl01 cloud-init[1053]: ci-info: +-------+--------------+--------------+---------------+-----------+-------+
Oct 26 13:50:17 ctl01 cloud-init[1053]: ci-info: +++++++++++++++++++Route IPv6 info+++++++++++++++++++
Oct 26 13:50:17 ctl01 cloud-init[1053]: ci-info: +-------+-------------+---------+-----------+-------+
Oct 26 13:50:17 ctl01 cloud-init[1053]: ci-info: | Route | Destination | Gateway | Interface | Flags |
Oct 26 13:50:17 ctl01 cloud-init[1053]: ci-info: +-------+-------------+---------+-----------+-------+
Oct 26 13:50:17 ctl01 cloud-init[1053]: ci-info: |   0   |  fe80::/64  |    ::   |    ens3   |   U   |
Oct 26 13:50:17 ctl01 cloud-init[1053]: ci-info: |   2   |    local    |    ::   |    ens3   |   U   |
Oct 26 13:50:17 ctl01 cloud-init[1053]: ci-info: |   3   |   ff00::/8  |    ::   |    ens3   |   U   |
Oct 26 13:50:17 ctl01 cloud-init[1053]: ci-info: +-------+-------------+---------+-----------+-------+
Oct 26 13:50:17 ctl01 cloud-init[1053]: Generating public/private rsa key pair.
Oct 26 13:50:17 ctl01 cloud-init[1053]: Your identification has been saved in /etc/ssh/ssh_host_rsa_key.
Oct 26 13:50:17 ctl01 cloud-init[1053]: Your public key has been saved in /etc/ssh/ssh_host_rsa_key.pub.
Oct 26 13:50:17 ctl01 cloud-init[1053]: The key fingerprint is:
Oct 26 13:50:17 ctl01 cloud-init[1053]: SHA256:Bxmm8ocwqIOMpCAcEDLwqPt2jY88R7Hz0k+8p7M2/a0 root@ctl01
Oct 26 13:50:17 ctl01 cloud-init[1053]: The key's randomart image is:
Oct 26 13:50:17 ctl01 cloud-init[1053]: +---[RSA 2048]----+
Oct 26 13:50:17 ctl01 cloud-init[1053]: |Oo      o        |
Oct 26 13:50:17 ctl01 cloud-init[1053]: |o+..   o o       |
Oct 26 13:50:17 ctl01 cloud-init[1053]: |+oo + . o        |
Oct 26 13:50:17 ctl01 cloud-init[1053]: |X.   =.. .       |
Oct 26 13:50:17 ctl01 cloud-init[1053]: |B.    ooS .      |
Oct 26 13:50:17 ctl01 cloud-init[1053]: | o    +. o       |
Oct 26 13:50:17 ctl01 cloud-init[1053]: |.    + +  o.     |
Oct 26 13:50:17 ctl01 cloud-init[1053]: | ...+.+ o.+.o  . |
Oct 26 13:50:17 ctl01 cloud-init[1053]: | ...o+.. o=* .E..|
Oct 26 13:50:17 ctl01 cloud-init[1053]: +----[SHA256]-----+
Oct 26 13:50:17 ctl01 cloud-init[1053]: Generating public/private dsa key pair.
Oct 26 13:50:17 ctl01 cloud-init[1053]: Your identification has been saved in /etc/ssh/ssh_host_dsa_key.
Oct 26 13:50:17 ctl01 cloud-init[1053]: Your public key has been saved in /etc/ssh/ssh_host_dsa_key.pub.
Oct 26 13:50:17 ctl01 cloud-init[1053]: The key fingerprint is:
Oct 26 13:50:17 ctl01 cloud-init[1053]: SHA256:cDHoMyO/2NICI644Et1Av1Mvk0oPZkCzXDldLkmVK2I root@ctl01
Oct 26 13:50:17 ctl01 cloud-init[1053]: The key's randomart image is:
Oct 26 13:50:17 ctl01 cloud-init[1053]: +---[DSA 1024]----+
Oct 26 13:50:17 ctl01 cloud-init[1053]: |     o.+=.       |
Oct 26 13:50:17 ctl01 cloud-init[1053]: |  + +.oo.o       |
Oct 26 13:50:17 ctl01 cloud-init[1053]: | + = o+ o.       |
Oct 26 13:50:17 ctl01 cloud-init[1053]: |  = E B+.        |
Oct 26 13:50:17 ctl01 cloud-init[1053]: | . = * BS        |
Oct 26 13:50:17 ctl01 cloud-init[1053]: |o + O = .        |
Oct 26 13:50:17 ctl01 cloud-init[1053]: |.o * O +         |
Oct 26 13:50:17 ctl01 cloud-init[1053]: |+.  = =          |
Oct 26 13:50:17 ctl01 cloud-init[1053]: |=.   o           |
Oct 26 13:50:17 ctl01 cloud-init[1053]: +----[SHA256]-----+
Oct 26 13:50:17 ctl01 cloud-init[1053]: Generating public/private ecdsa key pair.
Oct 26 13:50:17 ctl01 cloud-init[1053]: Your identification has been saved in /etc/ssh/ssh_host_ecdsa_key.
Oct 26 13:50:17 ctl01 cloud-init[1053]: Your public key has been saved in /etc/ssh/ssh_host_ecdsa_key.pub.
Oct 26 13:50:17 ctl01 cloud-init[1053]: The key fingerprint is:
Oct 26 13:50:17 ctl01 cloud-init[1053]: SHA256:BR7bSxv10GM4jWoAdrXLMd/rq6w+ZgfGIMbdTZfwDso root@ctl01
Oct 26 13:50:17 ctl01 cloud-init[1053]: The key's randomart image is:
Oct 26 13:50:17 ctl01 cloud-init[1053]: +---[ECDSA 256]---+
Oct 26 13:50:17 ctl01 cloud-init[1053]: |      o.+.. +* . |
Oct 26 13:50:17 ctl01 cloud-init[1053]: |     . o.= o=+B  |
Oct 26 13:50:17 ctl01 cloud-init[1053]: |     . .ooO+.+o. |
Oct 26 13:50:17 ctl01 cloud-init[1053]: |      + o=+X.+   |
Oct 26 13:50:17 ctl01 cloud-init[1053]: |     . .S+E . o  |
Oct 26 13:50:17 ctl01 cloud-init[1053]: |          +    . |
Oct 26 13:50:17 ctl01 cloud-init[1053]: |         . .  .  |
Oct 26 13:50:17 ctl01 cloud-init[1053]: |          +...   |
Oct 26 13:50:17 ctl01 cloud-init[1053]: |         +o+o.o. |
Oct 26 13:50:17 ctl01 cloud-init[1053]: +----[SHA256]-----+
Oct 26 13:50:17 ctl01 cloud-init[1053]: Generating public/private ed25519 key pair.
Oct 26 13:50:17 ctl01 cloud-init[1053]: Your identification has been saved in /etc/ssh/ssh_host_ed25519_key.
Oct 26 13:50:17 ctl01 cloud-init[1053]: Your public key has been saved in /etc/ssh/ssh_host_ed25519_key.pub.
Oct 26 13:50:17 ctl01 cloud-init[1053]: The key fingerprint is:
Oct 26 13:50:17 ctl01 cloud-init[1053]: SHA256:I4FIdY3NzdP+bMoGMpcLMrhTuKUaz7U801hMtYXTMoc root@ctl01
Oct 26 13:50:17 ctl01 cloud-init[1053]: The key's randomart image is:
Oct 26 13:50:17 ctl01 cloud-init[1053]: +--[ED25519 256]--+
Oct 26 13:50:17 ctl01 cloud-init[1053]: |  ... .= o =     |
Oct 26 13:50:17 ctl01 cloud-init[1053]: | . . o. + E =    |
Oct 26 13:50:17 ctl01 cloud-init[1053]: |  . . .  . X     |
Oct 26 13:50:17 ctl01 cloud-init[1053]: |       .. . .    |
Oct 26 13:50:17 ctl01 cloud-init[1053]: |     o.oS  . o   |
Oct 26 13:50:17 ctl01 cloud-init[1053]: |    o =.*.+   +  |
Oct 26 13:50:17 ctl01 cloud-init[1053]: |  .  *.* = + o   |
Oct 26 13:50:17 ctl01 cloud-init[1053]: |   +=o+.. . +    |
Oct 26 13:50:17 ctl01 cloud-init[1053]: |  ..o.oo   .     |
Oct 26 13:50:17 ctl01 cloud-init[1053]: +----[SHA256]-----+
Oct 26 13:50:17 ctl01 systemd[1]: Started Initial cloud-init job (metadata service crawler).
Oct 26 13:50:17 ctl01 systemd[1]: Reached target System Initialization.
Oct 26 13:50:17 ctl01 systemd[1]: Started Daily apt download activities.
Oct 26 13:50:17 ctl01 systemd[1]: Started Daily Cleanup of Temporary Directories.
Oct 26 13:50:17 ctl01 systemd[1]: Starting Socket activation for snappy daemon.
Oct 26 13:50:17 ctl01 systemd[1]: Starting LXD - unix socket.
Oct 26 13:50:17 ctl01 systemd[1]: Listening on D-Bus System Message Bus Socket.
Oct 26 13:50:17 ctl01 systemd[1]: Started Daily apt upgrade and clean activities.
Oct 26 13:50:17 ctl01 systemd[1]: Listening on UUID daemon activation socket.
Oct 26 13:50:17 ctl01 systemd[1]: Listening on ACPID Listen Socket.
Oct 26 13:50:17 ctl01 systemd[1]: Started ACPI Events Check.
Oct 26 13:50:17 ctl01 systemd[1]: Reached target Paths.
Oct 26 13:50:17 ctl01 systemd[1]: Started Message of the Day.
Oct 26 13:50:17 ctl01 systemd[1]: Reached target Timers.
Oct 26 13:50:17 ctl01 systemd[1]: Reached target Network is Online.
Oct 26 13:50:17 ctl01 systemd[1]: Starting iSCSI initiator daemon (iscsid)...
Oct 26 13:50:17 ctl01 systemd[1]: Reached target Cloud-config availability.
Oct 26 13:50:17 ctl01 systemd[1]: Listening on Socket activation for snappy daemon.
Oct 26 13:50:17 ctl01 systemd[1]: Listening on LXD - unix socket.
Oct 26 13:50:17 ctl01 systemd[1]: Reached target Sockets.
Oct 26 13:50:17 ctl01 systemd[1]: Reached target Basic System.
Oct 26 13:50:17 ctl01 systemd[1]: Started ACPI event daemon.
Oct 26 13:50:17 ctl01 systemd[1]: Started FUSE filesystem for LXC.
Oct 26 13:50:17 ctl01 systemd[1]: Starting Login Service...
Oct 26 13:50:17 ctl01 systemd[1]: Started Deferred execution scheduler.
Oct 26 13:50:17 ctl01 systemd[1]: Starting System Logging Service...
Oct 26 13:50:17 ctl01 systemd[1]: Starting Snappy daemon...
Oct 26 13:50:17 ctl01 systemd[1]: Starting /etc/rc.local Compatibility...
Oct 26 13:50:17 ctl01 systemd[1]: Starting LSB: MD monitoring daemon...
Oct 26 13:50:17 ctl01 systemd[1]: Starting LSB: Record successful boot for GRUB...
Oct 26 13:50:17 ctl01 systemd[1]: Starting Thermal Daemon Service...
Oct 26 13:50:17 ctl01 systemd[1]: Starting Pollinate to seed the pseudo random number generator...
Oct 26 13:50:17 ctl01 systemd[1]: Starting Accounts Service...
Oct 26 13:50:17 ctl01 systemd[1]: Started D-Bus System Message Bus.
Oct 26 13:50:17 ctl01 lxcfs[1167]: hierarchies:
Oct 26 13:50:17 ctl01 lxcfs[1167]:   0: fd:   5: rdma
Oct 26 13:50:17 ctl01 lxcfs[1167]:   1: fd:   6: memory
Oct 26 13:50:17 ctl01 lxcfs[1167]:   2: fd:   7: hugetlb
Oct 26 13:50:17 ctl01 lxcfs[1167]:   3: fd:   8: perf_event
Oct 26 13:50:17 ctl01 lxcfs[1167]:   4: fd:   9: blkio
Oct 26 13:50:17 ctl01 lxcfs[1167]:   5: fd:  10: cpuset
Oct 26 13:50:17 ctl01 lxcfs[1167]:   6: fd:  11: devices
Oct 26 13:50:17 ctl01 lxcfs[1167]:   7: fd:  12: pids
Oct 26 13:50:17 ctl01 lxcfs[1167]:   8: fd:  13: freezer
Oct 26 13:50:17 ctl01 lxcfs[1167]:   9: fd:  14: net_cls,net_prio
Oct 26 13:50:17 ctl01 lxcfs[1167]:  10: fd:  15: cpu,cpuacct
Oct 26 13:50:17 ctl01 lxcfs[1167]:  11: fd:  16: name=systemd
Oct 26 13:50:17 ctl01 acpid: starting up with netlink and the input layer
Oct 26 13:50:17 ctl01 acpid: 1 rule loaded
Oct 26 13:50:17 ctl01 acpid: waiting for events: event logging is off
Oct 26 13:50:18 ctl01 pollinate[1237]: client sent challenge to [https://entropy.ubuntu.com/]
Oct 26 13:50:18 ctl01 dbus[1257]: [system] AppArmor D-Bus mediation is enabled
Oct 26 13:50:18 ctl01 systemd[1]: Started Regular background program processing daemon.
Oct 26 13:50:18 ctl01 systemd[1]: Starting LXD - container startup/shutdown...
Oct 26 13:50:18 ctl01 systemd[1]: Starting The Salt Minion...
Oct 26 13:50:18 ctl01 cron[1331]: (CRON) INFO (pidfile fd = 3)
Oct 26 13:50:18 ctl01 systemd[1]: Started System Logging Service.
Oct 26 13:50:18 ctl01 systemd[1]: Started /etc/rc.local Compatibility.
Oct 26 13:50:18 ctl01 thermald[1227]: NO RAPL sysfs present
Oct 26 13:50:18 ctl01 thermald[1227]: 13 CPUID levels; family:model:stepping 0x6:3e:4 (6:62:4)
Oct 26 13:50:18 ctl01 thermald[1227]:  Need Linux PowerCap sysfs
Oct 26 13:50:18 ctl01 systemd[1]: Started Thermal Daemon Service.
Oct 26 13:50:18 ctl01 mdadm[1213]:  * Starting MD monitoring service mdadm --monitor
Oct 26 13:50:18 ctl01 systemd[1]: Started Login Service.
Oct 26 13:50:18 ctl01 systemd[1]: Started Unattended Upgrades Shutdown.
Oct 26 13:50:18 ctl01 cron[1331]: (CRON) INFO (Running @reboot jobs)
Oct 26 13:50:18 ctl01 iscsid: iSCSI logger with pid=1379 started!
Oct 26 13:50:18 ctl01 systemd[1]: Started iSCSI initiator daemon (iscsid).
Oct 26 13:50:18 ctl01 systemd[1]: Starting Login to default iSCSI targets...
Oct 26 13:50:18 ctl01 dbus[1257]: [system] Activating via systemd: service name='org.freedesktop.PolicyKit1' unit='polkitd.service'
Oct 26 13:50:18 ctl01 mdadm[1213]:    ...done.
Oct 26 13:50:18 ctl01 systemd[1]: Started LSB: MD monitoring daemon.
Oct 26 13:50:18 ctl01 thermald[1227]: Unsupported cpu model, use thermal-conf.xml file or run with --ignore-cpuid-check
Oct 26 13:50:18 ctl01 thermald[1227]: THD engine start failed
Oct 26 13:50:18 ctl01 systemd[1]: Starting Authenticate and Authorize Users to Run Privileged Tasks...
Oct 26 13:50:18 ctl01 iscsiadm[1402]: iscsiadm: No records found
Oct 26 13:50:18 ctl01 systemd[1]: Started Login to default iSCSI targets.
Oct 26 13:50:18 ctl01 systemd[1]: Reached target Remote File Systems (Pre).
Oct 26 13:50:18 ctl01 systemd[1]: Reached target Remote File Systems.
Oct 26 13:50:18 ctl01 systemd[1]: Starting LSB: Set the CPU Frequency Scaling governor to "ondemand"...
Oct 26 13:50:18 ctl01 systemd[1]: Starting LSB: daemon to balance interrupts for SMP systems...
Oct 26 13:50:18 ctl01 systemd[1]: Starting Permit User Sessions...
Oct 26 13:50:18 ctl01 systemd[1]: Starting LSB: automatic crash report generation...
Oct 26 13:50:18 ctl01 systemd[1]: Started LSB: Set the CPU Frequency Scaling governor to "ondemand".
Oct 26 13:50:18 ctl01 systemd[1]: Started Permit User Sessions.
Oct 26 13:50:18 ctl01 systemd[1]: Starting Hold until boot process finishes up...
Oct 26 13:50:18 ctl01 systemd[1]: Starting Terminate Plymouth Boot Screen...
Oct 26 13:50:18 ctl01 systemd[1]: Started Hold until boot process finishes up.
Oct 26 13:50:18 ctl01 systemd[1]: Started Serial Getty on ttyS0.
Oct 26 13:50:18 ctl01 systemd[1]: Started Getty on tty1.
Oct 26 13:50:18 ctl01 systemd[1]: Reached target Login Prompts.
Oct 26 13:50:18 ctl01 irqbalance[1431]:  * Starting SMP IRQ Balancer: irqbalance
Oct 26 13:50:18 ctl01 systemd[1]: Starting Set console scheme...
Oct 26 13:50:18 ctl01 systemd[1]: Started LSB: Record successful boot for GRUB.
Oct 26 13:50:18 ctl01 systemd[1]: Started Terminate Plymouth Boot Screen.
Oct 26 13:50:18 ctl01 snapd[1198]: AppArmor status: apparmor is enabled and all features are available
Oct 26 13:50:18 ctl01 polkitd[1407]: started daemon version 0.105 using authority implementation `local' version `0.105'
Oct 26 13:50:18 ctl01 dbus[1257]: [system] Successfully activated service 'org.freedesktop.PolicyKit1'
Oct 26 13:50:18 ctl01 accounts-daemon[1248]: started daemon version 0.6.40
Oct 26 13:50:18 ctl01 systemd[1]: Started Set console scheme.
Oct 26 13:50:18 ctl01 systemd[1]: Started Authenticate and Authorize Users to Run Privileged Tasks.
Oct 26 13:50:18 ctl01 irqbalance[1431]:    ...done.
Oct 26 13:50:18 ctl01 systemd[1]: Started LSB: daemon to balance interrupts for SMP systems.
Oct 26 13:50:18 ctl01 systemd[1]: Started Accounts Service.
Oct 26 13:50:18 ctl01 apport[1440]:  * Starting automatic crash report generation: apport
Oct 26 13:50:18 ctl01 apport[1440]:    ...done.
Oct 26 13:50:18 ctl01 systemd[1]: Started LSB: automatic crash report generation.
Oct 26 13:50:19 ctl01 snapd[1198]: helpers.go:145: error trying to compare the snap system key: system-key missing on disk
Oct 26 13:50:19 ctl01 systemd[1]: Started LXD - container startup/shutdown.
Oct 26 13:50:19 ctl01 snapd[1198]: daemon.go:338: started snapd/2.40 (series 16; classic) ubuntu/16.04 (amd64) linux/4.15.0-66-generic.
Oct 26 13:50:19 ctl01 systemd[1]: Started Snappy daemon.
Oct 26 13:50:19 ctl01 systemd[1]: Starting Wait until snapd is fully seeded...
Oct 26 13:50:19 ctl01 iscsid: iSCSI daemon with pid=1381 started!
Oct 26 13:50:19 ctl01 pollinate[1237]: client verified challenge/response with [https://entropy.ubuntu.com/]
Oct 26 13:50:19 ctl01 pollinate[1237]: client hashed response from [https://entropy.ubuntu.com/]
Oct 26 13:50:19 ctl01 pollinate[1237]: client successfully seeded [/dev/urandom]
Oct 26 13:50:19 ctl01 systemd[1]: Started Pollinate to seed the pseudo random number generator.
Oct 26 13:50:19 ctl01 systemd[1]: Starting OpenBSD Secure Shell server...
Oct 26 13:50:19 ctl01 systemd[1]: Started OpenBSD Secure Shell server.
Oct 26 13:50:19 ctl01 systemd[1]: Started The Salt Minion.
Oct 26 13:50:20 ctl01 systemd[1]: Started Wait until snapd is fully seeded.
Oct 26 13:50:20 ctl01 systemd[1]: Starting Apply the settings specified in cloud-config...
Oct 26 13:50:20 ctl01 systemd[1]: Reached target Multi-User System.
Oct 26 13:50:20 ctl01 systemd[1]: Reached target Graphical Interface.
Oct 26 13:50:20 ctl01 systemd[1]: Starting Update UTMP about System Runlevel Changes...
Oct 26 13:50:20 ctl01 systemd[1]: Started Update UTMP about System Runlevel Changes.
Oct 26 13:50:21 ctl01 salt-minion[1345]: [ERROR   ] DNS lookup or connection check of 'salt' failed.
Oct 26 13:50:21 ctl01 salt-minion[1345]: [ERROR   ] Master hostname: 'salt' not found or not responsive. Retrying in 30 seconds
Oct 26 13:50:21 ctl01 cloud-init[1718]: Generating locales (this might take a while)...
Oct 26 13:50:22 ctl01 cloud-init[1718]:   en_US.UTF-8... done
Oct 26 13:50:22 ctl01 cloud-init[1718]: Generation complete.
Oct 26 13:50:23 ctl01 cloud-init[1718]: Cloud-init v. 19.2-36-g059d049c-0ubuntu2~16.04.1 running 'modules:config' at Sat, 26 Oct 2019 13:50:21 +0000. Up 14.55 seconds.
Oct 26 13:50:23 ctl01 systemd[1]: Started Apply the settings specified in cloud-config.
Oct 26 13:50:23 ctl01 systemd[1]: Starting Execute cloud user/final scripts...
Oct 26 13:50:23 ctl01 systemd[1]: Stopping The Salt Minion...
Oct 26 13:50:23 ctl01 salt-minion[1345]: [WARNING ] Minion received a SIGTERM. Exiting.
Oct 26 13:50:24 ctl01 snapd[1198]: daemon.go:576: gracefully waiting for running hooks
Oct 26 13:50:24 ctl01 snapd[1198]: daemon.go:578: done waiting for running hooks
Oct 26 13:50:24 ctl01 snapd[1198]: daemon stop requested to wait for socket activation
Oct 26 13:50:24 ctl01 salt-minion[1345]: The Salt Minion is shutdown. Minion received a SIGTERM. Exited.
Oct 26 13:50:24 ctl01 systemd[1]: Stopped The Salt Minion.
Oct 26 13:50:24 ctl01 systemd[1]: Starting The Salt Minion...
Oct 26 13:50:24 ctl01 systemd[1]: Started The Salt Minion.
Oct 26 13:50:24 ctl01 ec2: 
Oct 26 13:50:24 ctl01 ec2: #############################################################
Oct 26 13:50:24 ctl01 ec2: -----BEGIN SSH HOST KEY FINGERPRINTS-----
Oct 26 13:50:24 ctl01 ec2: 1024 SHA256:cDHoMyO/2NICI644Et1Av1Mvk0oPZkCzXDldLkmVK2I root@ctl01 (DSA)
Oct 26 13:50:24 ctl01 ec2: 256 SHA256:BR7bSxv10GM4jWoAdrXLMd/rq6w+ZgfGIMbdTZfwDso root@ctl01 (ECDSA)
Oct 26 13:50:24 ctl01 ec2: 256 SHA256:I4FIdY3NzdP+bMoGMpcLMrhTuKUaz7U801hMtYXTMoc root@ctl01 (ED25519)
Oct 26 13:50:24 ctl01 ec2: 2048 SHA256:Bxmm8ocwqIOMpCAcEDLwqPt2jY88R7Hz0k+8p7M2/a0 root@ctl01 (RSA)
Oct 26 13:50:24 ctl01 ec2: -----END SSH HOST KEY FINGERPRINTS-----
Oct 26 13:50:24 ctl01 ec2: #############################################################
Oct 26 13:50:24 ctl01 cloud-init[1825]: Cloud-init v. 19.2-36-g059d049c-0ubuntu2~16.04.1 running 'modules:final' at Sat, 26 Oct 2019 13:50:23 +0000. Up 17.15 seconds.
Oct 26 13:50:24 ctl01 cloud-init[1825]: Cloud-init v. 19.2-36-g059d049c-0ubuntu2~16.04.1 finished at Sat, 26 Oct 2019 13:50:24 +0000. Datasource DataSourceNoCloud [seed=/dev/sr0][dsmode=net].  Up 18.03 seconds
Oct 26 13:50:24 ctl01 systemd[1]: Started Execute cloud user/final scripts.
Oct 26 13:50:24 ctl01 systemd[1]: Reached target Cloud-init target.
Oct 26 13:50:24 ctl01 systemd[1]: Startup finished in 3.327s (kernel) + 14.756s (userspace) = 18.084s.
Oct 26 13:50:41 ctl01 systemd[1]: Time has been changed
Oct 26 13:50:41 ctl01 systemd-timesyncd[711]: Synchronized to time server 91.189.89.199:123 (ntp.ubuntu.com).
Oct 26 13:50:57 ctl01 kernel: [   50.408987] random: crng init done
Oct 26 13:50:57 ctl01 kernel: [   50.408995] random: 7 urandom warning(s) missed due to ratelimiting
Oct 26 13:55:57 ctl01 salt-minion[1882]: [WARNING ] The function "module.run" is using its deprecated version and will expire in version "Sodium".
Oct 26 13:56:03 ctl01 systemd[1]: Started /usr/bin/apt-get -q -y remove telnet.
Oct 26 13:56:07 ctl01 systemd[1]: Started /usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold -o DPkg::Options::=--force-confdef install smartmontools.
Oct 26 13:56:10 ctl01 systemd[1]: Reloading.
Oct 26 13:56:10 ctl01 systemd[1]: Started ACPI event daemon.
Oct 26 13:56:11 ctl01 systemd[1]: Reloading.
Oct 26 13:56:11 ctl01 systemd[1]: Started ACPI event daemon.
Oct 26 13:56:11 ctl01 systemd[1]: Reloading.
Oct 26 13:56:11 ctl01 systemd[1]: Started ACPI event daemon.
Oct 26 13:56:11 ctl01 systemd[1]: Reloading.
Oct 26 13:56:11 ctl01 systemd[1]: Started ACPI event daemon.
Oct 26 13:56:11 ctl01 systemd[1]: Started Self Monitoring and Reporting Technology (SMART) Daemon.
Oct 26 13:56:11 ctl01 smartd[4136]: smartd 6.5 2016-01-24 r4214 [x86_64-linux-4.15.0-66-generic] (local build)
Oct 26 13:56:11 ctl01 smartd[4136]: Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org
Oct 26 13:56:11 ctl01 smartd[4136]: Opened configuration file /etc/smartd.conf
Oct 26 13:56:11 ctl01 smartd[4136]: Drive: DEVICESCAN, implied '-a' Directive on line 21 of file /etc/smartd.conf
Oct 26 13:56:11 ctl01 smartd[4136]: Configuration file /etc/smartd.conf was parsed, found DEVICESCAN, scanning devices
Oct 26 13:56:11 ctl01 smartd[4136]: Problem creating device name scan list
Oct 26 13:56:11 ctl01 smartd[4136]: In the system's table of devices NO devices found to scan
Oct 26 13:56:11 ctl01 smartd[4136]: Unable to monitor any SMART enabled devices. Try debug (-d) option. Exiting...
Oct 26 13:56:11 ctl01 systemd[1]: smartd.service: Main process exited, code=exited, status=17/n/a
Oct 26 13:56:11 ctl01 systemd[1]: smartd.service: Unit entered failed state.
Oct 26 13:56:11 ctl01 systemd[1]: smartd.service: Failed with result 'exit-code'.
Oct 26 13:56:11 ctl01 systemd[1]: Reloading.
Oct 26 13:56:11 ctl01 systemd[1]: Started ACPI event daemon.
Oct 26 13:56:11 ctl01 systemd[1]: Started Self Monitoring and Reporting Technology (SMART) Daemon.
Oct 26 13:56:11 ctl01 smartd[4180]: smartd 6.5 2016-01-24 r4214 [x86_64-linux-4.15.0-66-generic] (local build)
Oct 26 13:56:11 ctl01 smartd[4180]: Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org
Oct 26 13:56:11 ctl01 smartd[4180]: Opened configuration file /etc/smartd.conf
Oct 26 13:56:11 ctl01 smartd[4180]: Drive: DEVICESCAN, implied '-a' Directive on line 21 of file /etc/smartd.conf
Oct 26 13:56:11 ctl01 smartd[4180]: Configuration file /etc/smartd.conf was parsed, found DEVICESCAN, scanning devices
Oct 26 13:56:11 ctl01 smartd[4180]: Problem creating device name scan list
Oct 26 13:56:11 ctl01 smartd[4180]: In the system's table of devices NO devices found to scan
Oct 26 13:56:11 ctl01 smartd[4180]: Unable to monitor any SMART enabled devices. Try debug (-d) option. Exiting...
Oct 26 13:56:11 ctl01 systemd[1]: smartd.service: Main process exited, code=exited, status=17/n/a
Oct 26 13:56:11 ctl01 systemd[1]: smartd.service: Unit entered failed state.
Oct 26 13:56:11 ctl01 systemd[1]: smartd.service: Failed with result 'exit-code'.
Oct 26 13:56:11 ctl01 systemd[1]: Reloading.
Oct 26 13:56:11 ctl01 systemd[1]: Started ACPI event daemon.
Oct 26 13:56:13 ctl01 dbus[1257]: [system] Activating via systemd: service name='org.freedesktop.timedate1' unit='dbus-org.freedesktop.timedate1.service'
Oct 26 13:56:13 ctl01 systemd[1]: Starting Time & Date Service...
Oct 26 13:56:13 ctl01 dbus[1257]: [system] Successfully activated service 'org.freedesktop.timedate1'
Oct 26 13:56:13 ctl01 systemd[1]: Started Time & Date Service.
Oct 26 13:56:14 ctl01 salt-minion[1882]: [WARNING ] State for file: /boot/grub/grub.cfg - Neither 'source' nor 'contents' nor 'contents_pillar' nor 'contents_grains' was defined, yet 'replace' was set to 'True'. As there is no source to replace the file with, 'replace' has been set to 'False' to avoid reading the file unnecessarily.
Oct 26 13:56:14 ctl01 kernel: [  367.739094] nf_conntrack version 0.5.0 (65536 buckets, 262144 max)
Oct 26 13:56:19 ctl01 systemd[1]: Started /usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold -o DPkg::Options::=--force-confdef install sysfsutils.
Oct 26 13:56:21 ctl01 systemd[1]: Reloading.
Oct 26 13:56:21 ctl01 systemd[1]: Started ACPI event daemon.
Oct 26 13:56:21 ctl01 systemd[1]: Reloading.
Oct 26 13:56:21 ctl01 systemd[1]: Started ACPI event daemon.
Oct 26 13:56:21 ctl01 systemd[1]: Reloading.
Oct 26 13:56:21 ctl01 systemd[1]: Started ACPI event daemon.
Oct 26 13:56:21 ctl01 systemd[1]: Starting LSB: Set sysfs variables from /etc/sysfs.conf...
Oct 26 13:56:21 ctl01 sysfsutils[5118]:  * Setting sysfs variables...
Oct 26 13:56:21 ctl01 sysfsutils[5118]:    ...done.
Oct 26 13:56:21 ctl01 systemd[1]: Started LSB: Set sysfs variables from /etc/sysfs.conf.
Oct 26 13:56:22 ctl01 systemd[1]: Reloading.
Oct 26 13:56:22 ctl01 systemd[1]: Started ACPI event daemon.
Oct 26 13:56:24 ctl01 systemd[1]: Started /bin/systemctl stop ondemand.service.
Oct 26 13:56:24 ctl01 systemd[1]: Stopping LSB: Set the CPU Frequency Scaling governor to "ondemand"...
Oct 26 13:56:24 ctl01 systemd[1]: Stopped LSB: Set the CPU Frequency Scaling governor to "ondemand".
Oct 26 13:56:24 ctl01 systemd[1]: Started /usr/sbin/update-rc.d -f ondemand remove.
Oct 26 13:56:24 ctl01 systemd[1]: Reloading.
Oct 26 13:56:24 ctl01 systemd[1]: Started ACPI event daemon.
Oct 26 13:56:24 ctl01 dbus[1257]: [system] Activating via systemd: service name='org.freedesktop.locale1' unit='dbus-org.freedesktop.locale1.service'
Oct 26 13:56:24 ctl01 systemd[1]: Starting Locale Service...
Oct 26 13:56:24 ctl01 dbus[1257]: [system] Successfully activated service 'org.freedesktop.locale1'
Oct 26 13:56:24 ctl01 systemd[1]: Started Locale Service.
Oct 26 13:56:25 ctl01 salt-minion[1882]: [WARNING ] The function "module.run" is using its deprecated version and will expire in version "Sodium".
Oct 26 13:56:25 ctl01 systemd[1]: Reloading.
Oct 26 13:56:25 ctl01 systemd[1]: Started ACPI event daemon.
Oct 26 13:56:25 ctl01 salt-minion[1882]: [WARNING ] State for file: /etc/shadow - Neither 'source' nor 'contents' nor 'contents_pillar' nor 'contents_grains' was defined, yet 'replace' was set to 'True'. As there is no source to replace the file with, 'replace' has been set to 'False' to avoid reading the file unnecessarily.
Oct 26 13:56:25 ctl01 salt-minion[1882]: [WARNING ] State for file: /etc/gshadow - Neither 'source' nor 'contents' nor 'contents_pillar' nor 'contents_grains' was defined, yet 'replace' was set to 'True'. As there is no source to replace the file with, 'replace' has been set to 'False' to avoid reading the file unnecessarily.
Oct 26 13:56:25 ctl01 salt-minion[1882]: [WARNING ] State for file: /etc/group- - Neither 'source' nor 'contents' nor 'contents_pillar' nor 'contents_grains' was defined, yet 'replace' was set to 'True'. As there is no source to replace the file with, 'replace' has been set to 'False' to avoid reading the file unnecessarily.
Oct 26 13:56:25 ctl01 salt-minion[1882]: [WARNING ] State for file: /etc/group - Neither 'source' nor 'contents' nor 'contents_pillar' nor 'contents_grains' was defined, yet 'replace' was set to 'True'. As there is no source to replace the file with, 'replace' has been set to 'False' to avoid reading the file unnecessarily.
Oct 26 13:56:25 ctl01 salt-minion[1882]: [WARNING ] State for file: /etc/passwd- - Neither 'source' nor 'contents' nor 'contents_pillar' nor 'contents_grains' was defined, yet 'replace' was set to 'True'. As there is no source to replace the file with, 'replace' has been set to 'False' to avoid reading the file unnecessarily.
Oct 26 13:56:25 ctl01 salt-minion[1882]: [WARNING ] State for file: /etc/passwd - Neither 'source' nor 'contents' nor 'contents_pillar' nor 'contents_grains' was defined, yet 'replace' was set to 'True'. As there is no source to replace the file with, 'replace' has been set to 'False' to avoid reading the file unnecessarily.
Oct 26 13:56:25 ctl01 salt-minion[1882]: [WARNING ] State for file: /etc/gshadow- - Neither 'source' nor 'contents' nor 'contents_pillar' nor 'contents_grains' was defined, yet 'replace' was set to 'True'. As there is no source to replace the file with, 'replace' has been set to 'False' to avoid reading the file unnecessarily.
Oct 26 13:56:25 ctl01 salt-minion[1882]: [WARNING ] State for file: /etc/shadow- - Neither 'source' nor 'contents' nor 'contents_pillar' nor 'contents_grains' was defined, yet 'replace' was set to 'True'. As there is no source to replace the file with, 'replace' has been set to 'False' to avoid reading the file unnecessarily.
Oct 26 13:56:25 ctl01 systemd[1]: Started /usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold -o DPkg::Options::=--force-confdef install openvswitch-switch bridge-utils.
Oct 26 13:56:28 ctl01 systemd[1]: Reloading.
Oct 26 13:56:28 ctl01 systemd[1]: Started ACPI event daemon.
Oct 26 13:56:28 ctl01 systemd[1]: Reloading.
Oct 26 13:56:28 ctl01 systemd[1]: Started ACPI event daemon.
Oct 26 13:56:28 ctl01 systemd[1]: Reloading.
Oct 26 13:56:28 ctl01 systemd[1]: Started ACPI event daemon.
Oct 26 13:56:28 ctl01 systemd[1]: Reloading.
Oct 26 13:56:29 ctl01 systemd[1]: Started ACPI event daemon.
Oct 26 13:56:29 ctl01 systemd[1]: Starting Open vSwitch Internal Unit...
Oct 26 13:56:29 ctl01 ovs-ctl[6123]:  * /etc/openvswitch/conf.db does not exist
Oct 26 13:56:29 ctl01 ovs-ctl[6123]:  * Creating empty database /etc/openvswitch/conf.db
Oct 26 13:56:29 ctl01 ovs-ctl[6123]:  * Starting ovsdb-server
Oct 26 13:56:29 ctl01 ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl --no-wait -- init -- set Open_vSwitch . db-version=7.12.1
Oct 26 13:56:29 ctl01 ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl --no-wait set Open_vSwitch . ovs-version=2.5.5 "external-ids:system-id=\"49a27502-2a2a-45e5-a45d-6bfe2da26bab\"" "system-type=\"Ubuntu\"" "system-version=\"16.04-xenial\""
Oct 26 13:56:29 ctl01 ovs-ctl[6123]:  * Configuring Open vSwitch system IDs
Oct 26 13:56:29 ctl01 ovs-ctl[6123]:  * Inserting openvswitch module
Oct 26 13:56:29 ctl01 kernel: [  382.283986] openvswitch: Open vSwitch switching datapath
Oct 26 13:56:29 ctl01 ovs-ctl[6123]:  * Starting ovs-vswitchd
Oct 26 13:56:29 ctl01 ovs-ctl[6123]:  * Enabling remote OVSDB managers
Oct 26 13:56:29 ctl01 systemd[1]: Started Open vSwitch Internal Unit.
Oct 26 13:56:29 ctl01 systemd[1]: Starting Open vSwitch...
Oct 26 13:56:29 ctl01 systemd[1]: Started Open vSwitch.
Oct 26 13:56:29 ctl01 systemd[1]: Reloading.
Oct 26 13:56:29 ctl01 systemd[1]: Started ACPI event daemon.
Oct 26 13:56:29 ctl01 systemd[1]: Reloading.
Oct 26 13:56:29 ctl01 systemd[1]: Started ACPI event daemon.
Oct 26 13:56:33 ctl01 systemd[1]: Started /usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold -o DPkg::Options::=--force-confdef install bridge-utils.
Oct 26 13:56:34 ctl01 systemd[1]: Reloading.
Oct 26 13:56:34 ctl01 systemd[1]: Started ACPI event daemon.
Oct 26 13:56:34 ctl01 systemd[1]: Started /bin/systemctl enable networking.service.
Oct 26 13:56:34 ctl01 systemd[1]: Reloading.
Oct 26 13:56:34 ctl01 systemd[1]: Started ACPI event daemon.
Oct 26 13:56:34 ctl01 systemd[1]: Reloading.
Oct 26 13:56:34 ctl01 systemd[1]: Started ACPI event daemon.
Oct 26 13:56:34 ctl01 systemd[1]: Reloading.
Oct 26 13:56:34 ctl01 systemd[1]: Started ACPI event daemon.
Oct 26 13:56:34 ctl01 salt-minion[1882]: [WARNING ] The network state sls is requiring a reboot of the system to properly apply network configuration.
Oct 26 13:56:34 ctl01 systemd[1]: Started /usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold -o DPkg::Options::=--force-confdef install vlan.
Oct 26 13:56:35 ctl01 kernel: [  388.262982] 8021q: 802.1Q VLAN Support v1.8
Oct 26 13:56:35 ctl01 kernel: [  388.262996] 8021q: adding VLAN 0 to HW filter on device ens3
Oct 26 13:56:35 ctl01 kernel: [  388.271902] 8021q: adding VLAN 0 to HW filter on device ens5
Oct 26 13:56:35 ctl01 systemd[1]: Reloading OpenBSD Secure Shell server.
Oct 26 13:56:35 ctl01 systemd[1]: Started ifup for ens5.1000.
Oct 26 13:56:35 ctl01 systemd[1]: Found device /sys/subsystem/net/devices/ens5.1000.
Oct 26 13:56:35 ctl01 sh[6806]: ifup: waiting for lock on /run/network/ifstate.ens5
Oct 26 13:56:35 ctl01 systemd[1]: Reloaded OpenBSD Secure Shell server.
Oct 26 13:56:35 ctl01 sh[6806]: ifup: interface ens5.1000 already configured
Oct 26 13:56:36 ctl01 salt-minion[1882]: [ERROR   ] Command '['umount', '/dev/shm']' failed with return code: 32
Oct 26 13:56:36 ctl01 salt-minion[1882]: [ERROR   ] stderr: umount: /dev/shm: target is busy
Oct 26 13:56:36 ctl01 salt-minion[1882]:         (In some cases useful info about processes that
Oct 26 13:56:36 ctl01 salt-minion[1882]:          use the device is found by lsof(8) or fuser(1).)
Oct 26 13:56:36 ctl01 salt-minion[1882]: [ERROR   ] retcode: 32
Oct 26 13:56:37 ctl01 systemd[1]: Stopping Time & Date Service...
Oct 26 13:56:37 ctl01 systemd[1]: Stopped target Graphical Interface.
Oct 26 13:56:37 ctl01 systemd[1]: Closed Load/Save RF Kill Switch Status /dev/rfkill Watch.
Oct 26 13:56:37 ctl01 systemd[1]: Stopping Accounts Service...
Oct 26 13:56:37 ctl01 systemd[1]: Stopped target Cloud-init target.
Oct 26 13:56:37 ctl01 systemd[1]: Stopped Execute cloud user/final scripts.
Oct 26 13:56:37 ctl01 systemd[1]: Stopped Apply the settings specified in cloud-config.
Oct 26 13:56:37 ctl01 systemd[1]: Stopped target Cloud-config availability.
Oct 26 13:56:37 ctl01 systemd[1]: Stopped target Timers.
Oct 26 13:56:37 ctl01 systemd[1]: Stopped Daily Cleanup of Temporary Directories.
Oct 26 13:56:37 ctl01 systemd[1]: Stopped Daily apt upgrade and clean activities.
Oct 26 13:56:37 ctl01 systemd[1]: Stopped Daily apt download activities.
Oct 26 13:56:37 ctl01 systemd[1]: Stopping Locale Service...
Oct 26 13:56:37 ctl01 systemd[1]: Stopping ACPI event daemon...
Oct 26 13:56:37 ctl01 systemd[1]: Stopped Message of the Day.
Oct 26 13:56:37 ctl01 systemd[1]: Stopped target Multi-User System.
Oct 26 13:56:37 ctl01 systemd[1]: Stopping D-Bus System Message Bus...
Oct 26 13:56:37 ctl01 systemd[1]: Stopping LSB: Record successful boot for GRUB...
Oct 26 13:56:37 ctl01 systemd[1]: Stopping The Salt Minion...
Oct 26 13:56:37 ctl01 rsyslogd: [origin software="rsyslogd" swVersion="8.16.0" x-pid="1185" x-info="http://www.rsyslog.com"] exiting on signal 15.
Oct 26 13:56:53 ctl01 rsyslogd: [origin software="rsyslogd" swVersion="8.16.0" x-pid="2012" x-info="http://www.rsyslog.com"] start
Oct 26 13:56:53 ctl01 rsyslogd-2222: command 'KLogPermitNonKernelFacility' is currently not permitted - did you already set it via a RainerScript command (v6+ config)? [v8.16.0 try http://www.rsyslog.com/e/2222 ]
Oct 26 13:56:53 ctl01 rsyslogd: rsyslogd's groupid changed to 108
Oct 26 13:56:53 ctl01 rsyslogd: rsyslogd's userid changed to 104
Oct 26 13:56:53 ctl01 rsyslogd-2039: Could not open output pipe '/dev/xconsole':: No such file or directory [v8.16.0 try http://www.rsyslog.com/e/2039 ]
Oct 26 13:56:53 ctl01 systemd-modules-load[455]: Inserted module 'iscsi_tcp'
Oct 26 13:56:53 ctl01 loadkeys[450]: Loading /etc/console-setup/cached.kmap.gz
Oct 26 13:56:53 ctl01 systemd[1]: Started Uncomplicated firewall.
Oct 26 13:56:53 ctl01 systemd[1]: Started Create list of required static device nodes for the current kernel.
Oct 26 13:56:53 ctl01 systemd[1]: Started Remount Root and Kernel File Systems.
Oct 26 13:56:53 ctl01 systemd[1]: Started Set console keymap.
Oct 26 13:56:53 ctl01 systemd[1]: Started Nameserver information manager.
Oct 26 13:56:53 ctl01 systemd-modules-load[455]: Inserted module 'ib_iser'
Oct 26 13:56:53 ctl01 systemd[1]: Started LVM2 metadata daemon.
Oct 26 13:56:53 ctl01 systemd[1]: Starting udev Coldplug all Devices...
Oct 26 13:56:53 ctl01 systemd[1]: Starting Load/Save Random Seed...
Oct 26 13:56:53 ctl01 systemd[1]: Starting Create Static Device Nodes in /dev...
Oct 26 13:56:53 ctl01 systemd[1]: Starting Flush Journal to Persistent Storage...
Oct 26 13:56:53 ctl01 kernel: [    0.000000] Linux version 4.15.0-66-generic (buildd@lgw01-amd64-050) (gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.10)) #75~16.04.1-Ubuntu SMP Tue Oct 1 14:01:08 UTC 2019 (Ubuntu 4.15.0-66.75~16.04.1-generic 4.15.18)
Oct 26 13:56:53 ctl01 kernel: [    0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-4.15.0-66-generic root=UUID=3dc7f99a-1e06-428f-86bf-f1f8932b4515 ro console=tty1 console=ttyS0
Oct 26 13:56:53 ctl01 systemd-modules-load[455]: Inserted module 'nf_conntrack'
Oct 26 13:56:53 ctl01 kernel: [    0.000000] KERNEL supported cpus:
Oct 26 13:56:53 ctl01 kernel: [    0.000000]   Intel GenuineIntel
Oct 26 13:56:53 ctl01 kernel: [    0.000000]   AMD AuthenticAMD
Oct 26 13:56:53 ctl01 systemd[1]: Started Load Kernel Modules.
Oct 26 13:56:53 ctl01 kernel: [    0.000000]   Centaur CentaurHauls
Oct 26 13:56:53 ctl01 kernel: [    0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
Oct 26 13:56:53 ctl01 kernel: [    0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
Oct 26 13:56:53 ctl01 kernel: [    0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
Oct 26 13:56:53 ctl01 kernel: [    0.000000] x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256
Oct 26 13:56:53 ctl01 systemd[1]: Starting Apply Kernel Variables...
Oct 26 13:56:53 ctl01 kernel: [    0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format.
Oct 26 13:56:53 ctl01 kernel: [    0.000000] e820: BIOS-provided physical RAM map:
Oct 26 13:56:53 ctl01 kernel: [    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
Oct 26 13:56:53 ctl01 kernel: [    0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
Oct 26 13:56:53 ctl01 systemd[1]: Mounting FUSE Control File System...
Oct 26 13:56:53 ctl01 kernel: [    0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
Oct 26 13:56:53 ctl01 kernel: [    0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffdefff] usable
Oct 26 13:56:53 ctl01 systemd[1]: Mounting Configuration File System...
Oct 26 13:56:53 ctl01 kernel: [    0.000000] BIOS-e820: [mem 0x00000000bffdf000-0x00000000bfffffff] reserved
Oct 26 13:56:53 ctl01 kernel: [    0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved
Oct 26 13:56:53 ctl01 kernel: [    0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
Oct 26 13:56:53 ctl01 kernel: [    0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000003bfffffff] usable
Oct 26 13:56:53 ctl01 kernel: [    0.000000] NX (Execute Disable) protection: active
Oct 26 13:56:53 ctl01 systemd[1]: Mounted Configuration File System.
Oct 26 13:56:53 ctl01 kernel: [    0.000000] SMBIOS 2.8 present.
Oct 26 13:56:53 ctl01 systemd[1]: Mounted FUSE Control File System.
Oct 26 13:56:53 ctl01 kernel: [    0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Ubuntu-1.8.2-1ubuntu1 04/01/2014
Oct 26 13:56:53 ctl01 kernel: [    0.000000] Hypervisor detected: KVM
Oct 26 13:56:53 ctl01 kernel: [    0.000000] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
Oct 26 13:56:53 ctl01 systemd[1]: Started Load/Save Random Seed.
Oct 26 13:56:53 ctl01 kernel: [    0.000000] e820: remove [mem 0x000a0000-0x000fffff] usable
Oct 26 13:56:53 ctl01 kernel: [    0.000000] e820: last_pfn = 0x3c0000 max_arch_pfn = 0x400000000
Oct 26 13:56:53 ctl01 kernel: [    0.000000] MTRR default type: write-back
Oct 26 13:56:53 ctl01 systemd[1]: Started Apply Kernel Variables.
Oct 26 13:56:53 ctl01 kernel: [    0.000000] MTRR fixed ranges enabled:
Oct 26 13:56:53 ctl01 kernel: [    0.000000]   00000-9FFFF write-back
Oct 26 13:56:53 ctl01 kernel: [    0.000000]   A0000-BFFFF uncachable
Oct 26 13:56:53 ctl01 kernel: [    0.000000]   C0000-FFFFF write-protect
Oct 26 13:56:53 ctl01 kernel: [    0.000000] MTRR variable ranges enabled:
Oct 26 13:56:53 ctl01 systemd[1]: Started Create Static Device Nodes in /dev.
Oct 26 13:56:53 ctl01 kernel: [    0.000000]   0 base 00C0000000 mask FFC0000000 uncachable
Oct 26 13:56:53 ctl01 kernel: [    0.000000]   1 disabled
Oct 26 13:56:53 ctl01 kernel: [    0.000000]   2 disabled
Oct 26 13:56:53 ctl01 kernel: [    0.000000]   3 disabled
Oct 26 13:56:53 ctl01 kernel: [    0.000000]   4 disabled
Oct 26 13:56:53 ctl01 kernel: [    0.000000]   5 disabled
Oct 26 13:56:53 ctl01 systemd[1]: Starting udev Kernel Device Manager...
Oct 26 13:56:53 ctl01 kernel: [    0.000000]   6 disabled
Oct 26 13:56:53 ctl01 kernel: [    0.000000]   7 disabled
Oct 26 13:56:53 ctl01 kernel: [    0.000000] x86/PAT: Configuration [0-7]: WB  WC  UC- UC  WB  WP  UC- WT  
Oct 26 13:56:53 ctl01 kernel: [    0.000000] e820: last_pfn = 0xbffdf max_arch_pfn = 0x400000000
Oct 26 13:56:53 ctl01 systemd[1]: Started Flush Journal to Persistent Storage.
Oct 26 13:56:53 ctl01 kernel: [    0.000000] found SMP MP-table at [mem 0x000f6560-0x000f656f]
Oct 26 13:56:53 ctl01 kernel: [    0.000000] Scanning 1 areas for low memory corruption
Oct 26 13:56:53 ctl01 systemd[1]: Started udev Coldplug all Devices.
Oct 26 13:56:53 ctl01 kernel: [    0.000000] Using GB pages for direct mapping
Oct 26 13:56:53 ctl01 kernel: [    0.000000] BRK [0x3b6f3f000, 0x3b6f3ffff] PGTABLE
Oct 26 13:56:53 ctl01 kernel: [    0.000000] BRK [0x3b6f40000, 0x3b6f40fff] PGTABLE
Oct 26 13:56:53 ctl01 kernel: [    0.000000] BRK [0x3b6f41000, 0x3b6f41fff] PGTABLE
Oct 26 13:56:53 ctl01 kernel: [    0.000000] BRK [0x3b6f42000, 0x3b6f42fff] PGTABLE
Oct 26 13:56:53 ctl01 systemd[1]: Started Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling.
Oct 26 13:56:53 ctl01 kernel: [    0.000000] BRK [0x3b6f43000, 0x3b6f43fff] PGTABLE
Oct 26 13:56:53 ctl01 kernel: [    0.000000] BRK [0x3b6f44000, 0x3b6f44fff] PGTABLE
Oct 26 13:56:53 ctl01 kernel: [    0.000000] RAMDISK: [mem 0x31726000-0x34b8afff]
Oct 26 13:56:53 ctl01 systemd[1]: Reached target Local File Systems (Pre).
Oct 26 13:56:53 ctl01 kernel: [    0.000000] ACPI: Early table checksum verification disabled
Oct 26 13:56:53 ctl01 kernel: [    0.000000] ACPI: RSDP 0x00000000000F6510 000014 (v00 BOCHS )
Oct 26 13:56:53 ctl01 kernel: [    0.000000] ACPI: RSDT 0x00000000BFFE14CC 000030 (v01 BOCHS  BXPCRSDT 00000001 BXPC 00000001)
Oct 26 13:56:53 ctl01 kernel: [    0.000000] ACPI: FACP 0x00000000BFFE0854 000074 (v01 BOCHS  BXPCFACP 00000001 BXPC 00000001)
Oct 26 13:56:53 ctl01 kernel: [    0.000000] ACPI: DSDT 0x00000000BFFDFC80 000BD4 (v01 BOCHS  BXPCDSDT 00000001 BXPC 00000001)
Oct 26 13:56:53 ctl01 kernel: [    0.000000] ACPI: FACS 0x00000000BFFDFC40 000040
Oct 26 13:56:53 ctl01 systemd[1]: Reached target Local File Systems.
Oct 26 13:56:53 ctl01 kernel: [    0.000000] ACPI: SSDT 0x00000000BFFE08C8 000B54 (v01 BOCHS  BXPCSSDT 00000001 BXPC 00000001)
Oct 26 13:56:53 ctl01 kernel: [    0.000000] ACPI: APIC 0x00000000BFFE141C 0000B0 (v01 BOCHS  BXPCAPIC 00000001 BXPC 00000001)
Oct 26 13:56:53 ctl01 kernel: [    0.000000] ACPI: Local APIC address 0xfee00000
Oct 26 13:56:53 ctl01 kernel: [    0.000000] No NUMA configuration found
Oct 26 13:56:53 ctl01 systemd[1]: Starting LSB: AppArmor initialization...
Oct 26 13:56:53 ctl01 kernel: [    0.000000] Faking a node at [mem 0x0000000000000000-0x00000003bfffffff]
Oct 26 13:56:53 ctl01 kernel: [    0.000000] NODE_DATA(0) allocated [mem 0x3bffd5000-0x3bfffffff]
Oct 26 13:56:53 ctl01 systemd[1]: Starting Initial cloud-init job (pre-networking)...
Oct 26 13:56:53 ctl01 systemd[1]: Starting Create Volatile Files and Directories...
Oct 26 13:56:53 ctl01 kernel: [    0.000000] kvm-clock: cpu 0, msr 3:bff54001, primary cpu clock
Oct 26 13:56:53 ctl01 kernel: [    0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00
Oct 26 13:56:53 ctl01 kernel: [    0.000000] kvm-clock: using sched offset of 396974691548 cycles
Oct 26 13:56:53 ctl01 kernel: [    0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns
Oct 26 13:56:53 ctl01 kernel: [    0.000000] Zone ranges:
Oct 26 13:56:53 ctl01 kernel: [    0.000000]   DMA      [mem 0x0000000000001000-0x0000000000ffffff]
Oct 26 13:56:53 ctl01 kernel: [    0.000000]   DMA32    [mem 0x0000000001000000-0x00000000ffffffff]
Oct 26 13:56:53 ctl01 kernel: [    0.000000]   Normal   [mem 0x0000000100000000-0x00000003bfffffff]
Oct 26 13:56:53 ctl01 kernel: [    0.000000]   Device   empty
Oct 26 13:56:53 ctl01 kernel: [    0.000000] Movable zone start for each node
Oct 26 13:56:53 ctl01 kernel: [    0.000000] Early memory node ranges
Oct 26 13:56:53 ctl01 kernel: [    0.000000]   node   0: [mem 0x0000000000001000-0x000000000009efff]
Oct 26 13:56:53 ctl01 kernel: [    0.000000]   node   0: [mem 0x0000000000100000-0x00000000bffdefff]
Oct 26 13:56:53 ctl01 kernel: [    0.000000]   node   0: [mem 0x0000000100000000-0x00000003bfffffff]
Oct 26 13:56:53 ctl01 kernel: [    0.000000] Reserved but unavailable: 98 pages
Oct 26 13:56:53 ctl01 kernel: [    0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x00000003bfffffff]
Oct 26 13:56:53 ctl01 kernel: [    0.000000] On node 0 totalpages: 3669885
Oct 26 13:56:53 ctl01 kernel: [    0.000000]   DMA zone: 64 pages used for memmap
Oct 26 13:56:53 ctl01 kernel: [    0.000000]   DMA zone: 21 pages reserved
Oct 26 13:56:53 ctl01 kernel: [    0.000000]   DMA zone: 3998 pages, LIFO batch:0
Oct 26 13:56:53 ctl01 kernel: [    0.000000]   DMA32 zone: 12224 pages used for memmap
Oct 26 13:56:53 ctl01 kernel: [    0.000000]   DMA32 zone: 782303 pages, LIFO batch:31
Oct 26 13:56:53 ctl01 kernel: [    0.000000]   Normal zone: 45056 pages used for memmap
Oct 26 13:56:53 ctl01 kernel: [    0.000000]   Normal zone: 2883584 pages, LIFO batch:31
Oct 26 13:56:53 ctl01 kernel: [    0.000000] ACPI: PM-Timer IO Port: 0x608
Oct 26 13:56:53 ctl01 kernel: [    0.000000] ACPI: Local APIC address 0xfee00000
Oct 26 13:56:53 ctl01 kernel: [    0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
Oct 26 13:56:53 ctl01 kernel: [    0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23
Oct 26 13:56:53 ctl01 kernel: [    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
Oct 26 13:56:53 ctl01 kernel: [    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
Oct 26 13:56:53 ctl01 kernel: [    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
Oct 26 13:56:53 ctl01 kernel: [    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
Oct 26 13:56:53 ctl01 kernel: [    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
Oct 26 13:56:53 ctl01 kernel: [    0.000000] ACPI: IRQ0 used by override.
Oct 26 13:56:53 ctl01 kernel: [    0.000000] ACPI: IRQ5 used by override.
Oct 26 13:56:53 ctl01 kernel: [    0.000000] ACPI: IRQ9 used by override.
Oct 26 13:56:53 ctl01 kernel: [    0.000000] ACPI: IRQ10 used by override.
Oct 26 13:56:53 ctl01 kernel: [    0.000000] ACPI: IRQ11 used by override.
Oct 26 13:56:53 ctl01 kernel: [    0.000000] Using ACPI (MADT) for SMP configuration information
Oct 26 13:56:53 ctl01 kernel: [    0.000000] smpboot: Allowing 8 CPUs, 0 hotplug CPUs
Oct 26 13:56:53 ctl01 kernel: [    0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff]
Oct 26 13:56:53 ctl01 kernel: [    0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff]
Oct 26 13:56:53 ctl01 kernel: [    0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff]
Oct 26 13:56:53 ctl01 kernel: [    0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff]
Oct 26 13:56:53 ctl01 kernel: [    0.000000] PM: Registered nosave memory: [mem 0xbffdf000-0xbfffffff]
Oct 26 13:56:53 ctl01 kernel: [    0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff]
Oct 26 13:56:53 ctl01 kernel: [    0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff]
Oct 26 13:56:53 ctl01 kernel: [    0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff]
Oct 26 13:56:53 ctl01 kernel: [    0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff]
Oct 26 13:56:53 ctl01 kernel: [    0.000000] e820: [mem 0xc0000000-0xfeffbfff] available for PCI devices
Oct 26 13:56:53 ctl01 kernel: [    0.000000] Booting paravirtualized kernel on KVM
Oct 26 13:56:53 ctl01 kernel: [    0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns
Oct 26 13:56:53 ctl01 kernel: [    0.000000] random: get_random_bytes called from start_kernel+0x99/0x51b with crng_init=0
Oct 26 13:56:53 ctl01 kernel: [    0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:8 nr_cpu_ids:8 nr_node_ids:1
Oct 26 13:56:53 ctl01 kernel: [    0.000000] percpu: Embedded 46 pages/cpu s151552 r8192 d28672 u262144
Oct 26 13:56:53 ctl01 systemd-tmpfiles[526]: [/usr/lib/tmpfiles.d/var.conf:14] Duplicate line for path "/var/log", ignoring.
Oct 26 13:56:53 ctl01 systemd[1]: Starting Tell Plymouth To Write Out Runtime Data...
Oct 26 13:56:53 ctl01 systemd[1]: Starting Set console font and keymap...
Oct 26 13:56:53 ctl01 systemd[1]: Started Tell Plymouth To Write Out Runtime Data.
Oct 26 13:56:53 ctl01 kernel: [    0.000000] pcpu-alloc: s151552 r8192 d28672 u262144 alloc=1*2097152
Oct 26 13:56:53 ctl01 kernel: [    0.000000] pcpu-alloc: [0] 0 1 2 3 4 5 6 7 
Oct 26 13:56:53 ctl01 systemd[1]: Started udev Kernel Device Manager.
Oct 26 13:56:53 ctl01 kernel: [    0.000000] KVM setup async PF for cpu 0
Oct 26 13:56:53 ctl01 kernel: [    0.000000] kvm-stealtime: cpu 0, msr 3bfc24040
Oct 26 13:56:53 ctl01 kernel: [    0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes)
Oct 26 13:56:53 ctl01 kernel: [    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 3612520
Oct 26 13:56:53 ctl01 systemd[1]: Started Dispatch Password Requests to Console Directory Watch.
Oct 26 13:56:53 ctl01 kernel: [    0.000000] Policy zone: Normal
Oct 26 13:56:53 ctl01 kernel: [    0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-4.15.0-66-generic root=UUID=3dc7f99a-1e06-428f-86bf-f1f8932b4515 ro console=tty1 console=ttyS0
Oct 26 13:56:53 ctl01 kernel: [    0.000000] Calgary: detecting Calgary via BIOS EBDA area
Oct 26 13:56:53 ctl01 kernel: [    0.000000] Calgary: Unable to locate Rio Grande table in EBDA - bailing!
Oct 26 13:56:53 ctl01 apparmor[523]:  * Starting AppArmor profiles
Oct 26 13:56:53 ctl01 kernel: [    0.000000] Memory: 14300308K/14679540K available (12300K kernel code, 2480K rwdata, 4208K rodata, 2428K init, 2388K bss, 379232K reserved, 0K cma-reserved)
Oct 26 13:56:53 ctl01 systemd[1]: Started Create Volatile Files and Directories.
Oct 26 13:56:53 ctl01 kernel: [    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=8, Nodes=1
Oct 26 13:56:53 ctl01 kernel: [    0.000000] Kernel/User page tables isolation: enabled
Oct 26 13:56:53 ctl01 kernel: [    0.000000] ftrace: allocating 39287 entries in 154 pages
Oct 26 13:56:53 ctl01 systemd[1]: Starting Network Time Synchronization...
Oct 26 13:56:53 ctl01 kernel: [    0.004000] Hierarchical RCU implementation.
Oct 26 13:56:53 ctl01 kernel: [    0.004000] 	RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=8.
Oct 26 13:56:53 ctl01 kernel: [    0.004000] 	Tasks RCU enabled.
Oct 26 13:56:53 ctl01 kernel: [    0.004000] RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=8
Oct 26 13:56:53 ctl01 systemd[1]: Starting Update UTMP about System Boot/Shutdown...
Oct 26 13:56:53 ctl01 kernel: [    0.004000] NR_IRQS: 524544, nr_irqs: 488, preallocated irqs: 16
Oct 26 13:56:53 ctl01 kernel: [    0.004000] Console: colour VGA+ 80x25
Oct 26 13:56:53 ctl01 kernel: [    0.004000] console [tty1] enabled
Oct 26 13:56:53 ctl01 kernel: [    0.004000] console [ttyS0] enabled
Oct 26 13:56:53 ctl01 kernel: [    0.004000] ACPI: Core revision 20170831
Oct 26 13:56:53 ctl01 kernel: [    0.004000] ACPI: 2 ACPI AML tables successfully acquired and loaded
Oct 26 13:56:53 ctl01 systemd[1]: Found device /dev/ttyS0.
Oct 26 13:56:53 ctl01 kernel: [    0.004005] APIC: Switch to symmetric I/O mode setup
Oct 26 13:56:53 ctl01 kernel: [    0.005243] x2apic enabled
Oct 26 13:56:53 ctl01 kernel: [    0.006123] Switched APIC routing to physical x2apic.
Oct 26 13:56:53 ctl01 kernel: [    0.008270] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
Oct 26 13:56:53 ctl01 kernel: [    0.009485] tsc: Detected 2799.994 MHz processor
Oct 26 13:56:53 ctl01 kernel: [    0.010396] Calibrating delay loop (skipped) preset value.. 5599.98 BogoMIPS (lpj=11199976)
Oct 26 13:56:53 ctl01 systemd[1]: Started Network Time Synchronization.
Oct 26 13:56:53 ctl01 kernel: [    0.012002] pid_max: default: 32768 minimum: 301
Oct 26 13:56:53 ctl01 kernel: [    0.013247] Security Framework initialized
Oct 26 13:56:53 ctl01 kernel: [    0.014153] Yama: becoming mindful.
Oct 26 13:56:53 ctl01 kernel: [    0.014905] AppArmor: AppArmor initialized
Oct 26 13:56:53 ctl01 kernel: [    0.018653] Dentry cache hash table entries: 2097152 (order: 12, 16777216 bytes)
Oct 26 13:56:53 ctl01 kernel: [    0.021206] Inode-cache hash table entries: 1048576 (order: 11, 8388608 bytes)
Oct 26 13:56:53 ctl01 systemd[1]: Started Update UTMP about System Boot/Shutdown.
Oct 26 13:56:53 ctl01 kernel: [    0.022654] Mount-cache hash table entries: 32768 (order: 6, 262144 bytes)
Oct 26 13:56:53 ctl01 kernel: [    0.024046] Mountpoint-cache hash table entries: 32768 (order: 6, 262144 bytes)
Oct 26 13:56:53 ctl01 kernel: [    0.025744] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0
Oct 26 13:56:53 ctl01 kernel: [    0.026779] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0
Oct 26 13:56:53 ctl01 kernel: [    0.028003] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization
Oct 26 13:56:53 ctl01 kernel: [    0.029625] Spectre V2 : Mitigation: Full generic retpoline
Oct 26 13:56:53 ctl01 kernel: [    0.030698] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch
Oct 26 13:56:53 ctl01 systemd[1]: Reached target System Time Synchronized.
Oct 26 13:56:53 ctl01 kernel: [    0.032001] Spectre V2 : Enabling Restricted Speculation for firmware calls
Oct 26 13:56:53 ctl01 kernel: [    0.033317] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier
Oct 26 13:56:53 ctl01 kernel: [    0.034933] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl and seccomp
Oct 26 13:56:53 ctl01 systemd[1]: Started Set console font and keymap.
Oct 26 13:56:53 ctl01 kernel: [    0.036030] MDS: Mitigation: Clear CPU buffers
Oct 26 13:56:53 ctl01 kernel: [    0.042697] Freeing SMP alternatives memory: 36K
Oct 26 13:56:53 ctl01 systemd[1]: Created slice system-getty.slice.
Oct 26 13:56:53 ctl01 kernel: [    0.045343] TSC deadline timer enabled
Oct 26 13:56:53 ctl01 kernel: [    0.045346] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz (family: 0x6, model: 0x3e, stepping: 0x4)
Oct 26 13:56:53 ctl01 kernel: [    0.047299] Performance Events: IvyBridge events, Intel PMU driver.
Oct 26 13:56:53 ctl01 kernel: [    0.048000] ... version:                2
Oct 26 13:56:53 ctl01 systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.
Oct 26 13:56:53 ctl01 kernel: [    0.048004] ... bit width:              48
Oct 26 13:56:53 ctl01 kernel: [    0.048836] ... generic registers:      4
Oct 26 13:56:53 ctl01 kernel: [    0.049647] ... value mask:             0000ffffffffffff
Oct 26 13:56:53 ctl01 apparmor[523]: Skipping profile in /etc/apparmor.d/disable: usr.sbin.rsyslogd
Oct 26 13:56:53 ctl01 kernel: [    0.050673] ... max period:             000000007fffffff
Oct 26 13:56:53 ctl01 systemd[1]: Found device Virtio network device.
Oct 26 13:56:53 ctl01 kernel: [    0.051714] ... fixed-purpose events:   3
Oct 26 13:56:53 ctl01 kernel: [    0.052003] ... event mask:             000000070000000f
Oct 26 13:56:53 ctl01 kernel: [    0.053078] Hierarchical SRCU implementation.
Oct 26 13:56:53 ctl01 kernel: [    0.054633] smp: Bringing up secondary CPUs ...
Oct 26 13:56:53 ctl01 kernel: [    0.055646] x86: Booting SMP configuration:
Oct 26 13:56:53 ctl01 kernel: [    0.056006] .... node  #0, CPUs:      #1
Oct 26 13:56:53 ctl01 kernel: [    0.004000] kvm-clock: cpu 1, msr 3:bff54041, secondary cpu clock
Oct 26 13:56:53 ctl01 kernel: [    0.060037] KVM setup async PF for cpu 1
Oct 26 13:56:53 ctl01 kernel: [    0.060997] kvm-stealtime: cpu 1, msr 3bfc64040
Oct 26 13:56:53 ctl01 kernel: [    0.062096]  #2
Oct 26 13:56:53 ctl01 kernel: [    0.004000] kvm-clock: cpu 2, msr 3:bff54081, secondary cpu clock
Oct 26 13:56:53 ctl01 kernel: [    0.064052] KVM setup async PF for cpu 2
Oct 26 13:56:53 ctl01 kernel: [    0.065128] kvm-stealtime: cpu 2, msr 3bfca4040
Oct 26 13:56:53 ctl01 systemd[1]: Found device Virtio network device.
Oct 26 13:56:53 ctl01 systemd-udevd[552]: Could not generate persistent MAC address for br-mgmt: No such file or directory
Oct 26 13:56:53 ctl01 kernel: [    0.066234]  #3
Oct 26 13:56:53 ctl01 kernel: [    0.004000] kvm-clock: cpu 3, msr 3:bff540c1, secondary cpu clock
Oct 26 13:56:53 ctl01 kernel: [    0.068031] KVM setup async PF for cpu 3
Oct 26 13:56:53 ctl01 systemd[1]: Found device /sys/subsystem/net/devices/br-mgmt.
Oct 26 13:56:53 ctl01 kernel: [    0.069020] kvm-stealtime: cpu 3, msr 3bfce4040
Oct 26 13:56:53 ctl01 kernel: [    0.070118]  #4
Oct 26 13:56:53 ctl01 kernel: [    0.004000] kvm-clock: cpu 4, msr 3:bff54101, secondary cpu clock
Oct 26 13:56:53 ctl01 systemd[1]: Found device Virtio network device.
Oct 26 13:56:53 ctl01 apparmor[523]:    ...done.
Oct 26 13:56:53 ctl01 kernel: [    0.072032] KVM setup async PF for cpu 4
Oct 26 13:56:53 ctl01 kernel: [    0.073015] kvm-stealtime: cpu 4, msr 3bfd24040
Oct 26 13:56:53 ctl01 kernel: [    0.074124]  #5
Oct 26 13:56:53 ctl01 kernel: [    0.004000] kvm-clock: cpu 5, msr 3:bff54141, secondary cpu clock
Oct 26 13:56:53 ctl01 kernel: [    0.076031] KVM setup async PF for cpu 5
Oct 26 13:56:53 ctl01 kernel: [    0.077068] kvm-stealtime: cpu 5, msr 3bfd64040
Oct 26 13:56:53 ctl01 systemd[1]: Started LSB: AppArmor initialization.
Oct 26 13:56:53 ctl01 kernel: [    0.078190]  #6
Oct 26 13:56:53 ctl01 kernel: [    0.004000] kvm-clock: cpu 6, msr 3:bff54181, secondary cpu clock
Oct 26 13:56:53 ctl01 cloud-init[524]: Cloud-init v. 19.2-36-g059d049c-0ubuntu2~16.04.1 running 'init-local' at Sat, 26 Oct 2019 13:56:50 +0000. Up 8.04 seconds.
Oct 26 13:56:53 ctl01 kernel: [    0.080027] KVM setup async PF for cpu 6
Oct 26 13:56:53 ctl01 kernel: [    0.082123] kvm-stealtime: cpu 6, msr 3bfda4040
Oct 26 13:56:53 ctl01 kernel: [    0.083009]  #7
Oct 26 13:56:53 ctl01 systemd[1]: Started Initial cloud-init job (pre-networking).
Oct 26 13:56:53 ctl01 kernel: [    0.004000] kvm-clock: cpu 7, msr 3:bff541c1, secondary cpu clock
Oct 26 13:56:53 ctl01 kernel: [    0.088034] KVM setup async PF for cpu 7
Oct 26 13:56:53 ctl01 systemd[1]: Reached target Network (Pre).
Oct 26 13:56:53 ctl01 kernel: [    0.089016] kvm-stealtime: cpu 7, msr 3bfde4040
Oct 26 13:56:53 ctl01 kernel: [    0.090135] smp: Brought up 1 node, 8 CPUs
Oct 26 13:56:53 ctl01 kernel: [    0.092006] smpboot: Max logical packages: 8
Oct 26 13:56:53 ctl01 kernel: [    0.092896] smpboot: Total of 8 processors activated (44799.90 BogoMIPS)
Oct 26 13:56:53 ctl01 kernel: [    0.095009] devtmpfs: initialized
Oct 26 13:56:53 ctl01 systemd[1]: Started ifup for ens3.
Oct 26 13:56:53 ctl01 kernel: [    0.095009] x86/mm: Memory block size: 128MB
Oct 26 13:56:53 ctl01 kernel: [    0.096973] evm: security.selinux
Oct 26 13:56:53 ctl01 kernel: [    0.097657] evm: security.SMACK64
Oct 26 13:56:53 ctl01 kernel: [    0.098366] evm: security.SMACK64EXEC
Oct 26 13:56:53 ctl01 kernel: [    0.099099] evm: security.SMACK64TRANSMUTE
Oct 26 13:56:53 ctl01 kernel: [    0.099898] evm: security.SMACK64MMAP
Oct 26 13:56:53 ctl01 kernel: [    0.100007] evm: security.apparmor
Oct 26 13:56:53 ctl01 kernel: [    0.100763] evm: security.ima
Oct 26 13:56:53 ctl01 kernel: [    0.101399] evm: security.capability
Oct 26 13:56:53 ctl01 kernel: [    0.102182] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
Oct 26 13:56:53 ctl01 systemd[1]: Starting Raise network interfaces...
Oct 26 13:56:53 ctl01 kernel: [    0.104016] futex hash table entries: 2048 (order: 5, 131072 bytes)
Oct 26 13:56:53 ctl01 kernel: [    0.105285] pinctrl core: initialized pinctrl subsystem
Oct 26 13:56:53 ctl01 systemd[1]: Started ifup for ens4.
Oct 26 13:56:53 ctl01 kernel: [    0.106451] RTC time: 13:56:43, date: 10/26/19
Oct 26 13:56:53 ctl01 kernel: [    0.108131] NET: Registered protocol family 16
Oct 26 13:56:53 ctl01 kernel: [    0.109098] audit: initializing netlink subsys (disabled)
Oct 26 13:56:53 ctl01 kernel: [    0.110155] audit: type=2000 audit(1572098202.122:1): state=initialized audit_enabled=0 res=1
Oct 26 13:56:53 ctl01 kernel: [    0.112011] cpuidle: using governor ladder
Oct 26 13:56:53 ctl01 systemd[1]: Started ifup for br-mgmt.
Oct 26 13:56:53 ctl01 kernel: [    0.112870] cpuidle: using governor menu
Oct 26 13:56:53 ctl01 kernel: [    0.113869] ACPI: bus type PCI registered
Oct 26 13:56:53 ctl01 kernel: [    0.114780] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
Oct 26 13:56:53 ctl01 kernel: [    0.116126] PCI: Using configuration type 1 for base access
Oct 26 13:56:53 ctl01 kernel: [    0.117204] core: PMU erratum BJ122, BV98, HSD29 workaround disabled, HT off
Oct 26 13:56:53 ctl01 kernel: [    0.120159] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
Oct 26 13:56:53 ctl01 systemd[1]: Started ifup for ens5.
Oct 26 13:56:53 ctl01 kernel: [    0.121284] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
Oct 26 13:56:53 ctl01 kernel: [    0.122605] ACPI: Added _OSI(Module Device)
Oct 26 13:56:53 ctl01 kernel: [    0.124005] ACPI: Added _OSI(Processor Device)
Oct 26 13:56:53 ctl01 kernel: [    0.124892] ACPI: Added _OSI(3.0 _SCP Extensions)
Oct 26 13:56:53 ctl01 kernel: [    0.125826] ACPI: Added _OSI(Processor Aggregator Device)
Oct 26 13:56:53 ctl01 kernel: [    0.126894] ACPI: Added _OSI(Linux-Dell-Video)
Oct 26 13:56:53 ctl01 sh[1159]: Waiting for br-mgmt to get ready (MAXWAIT is 32 seconds).
Oct 26 13:56:53 ctl01 kernel: [    0.127757] ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio)
Oct 26 13:56:53 ctl01 kernel: [    0.128004] ACPI: Added _OSI(Linux-HPI-Hybrid-Graphics)
Oct 26 13:56:53 ctl01 kernel: [    0.130375] ACPI: Interpreter enabled
Oct 26 13:56:53 ctl01 kernel: [    0.131127] ACPI: (supports S0 S5)
Oct 26 13:56:53 ctl01 kernel: [    0.131825] ACPI: Using IOAPIC for interrupt routing
Oct 26 13:56:53 ctl01 kernel: [    0.132018] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
Oct 26 13:56:53 ctl01 ifup[1215]: /sbin/ifup: waiting for lock on /run/network/ifstate.ens5
Oct 26 13:56:53 ctl01 kernel: [    0.134410] ACPI: Enabled 16 GPEs in block 00 to 0F
Oct 26 13:56:53 ctl01 kernel: [    0.138626] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
Oct 26 13:56:53 ctl01 kernel: [    0.139792] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI]
Oct 26 13:56:53 ctl01 sh[1165]: WARNING:  Could not open /proc/net/vlan/config.  Maybe you need to load the 8021q module, or maybe you are not using PROCFS??
Oct 26 13:56:53 ctl01 sh[1165]: Set name-type for VLAN subsystem. Should be visible in /proc/net/vlan/config
Oct 26 13:56:53 ctl01 sh[1165]: Added VLAN with VID == 1000 to IF -:ens5:-
Oct 26 13:56:53 ctl01 ifup[1215]: /sbin/ifup: waiting for lock on /run/network/ifstate.br-mgmt
Oct 26 13:56:53 ctl01 systemd[1]: Started ifup for ens5.1000.
Oct 26 13:56:53 ctl01 systemd[1]: Found device /sys/subsystem/net/devices/ens5.1000.
Oct 26 13:56:53 ctl01 sh[1586]: Set name-type for VLAN subsystem. Should be visible in /proc/net/vlan/config
Oct 26 13:56:53 ctl01 kernel: [    0.140009] acpi PNP0A03:00: _OSC failed (AE_NOT_FOUND); disabling ASPM
Oct 26 13:56:53 ctl01 kernel: [    0.141240] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge.
Oct 26 13:56:53 ctl01 systemd[1]: Started Raise network interfaces.
Oct 26 13:56:53 ctl01 kernel: [    0.143593] acpiphp: Slot [3] registered
Oct 26 13:56:53 ctl01 kernel: [    0.144046] acpiphp: Slot [4] registered
Oct 26 13:56:53 ctl01 kernel: [    0.144899] acpiphp: Slot [5] registered
Oct 26 13:56:53 ctl01 kernel: [    0.145739] acpiphp: Slot [6] registered
Oct 26 13:56:53 ctl01 kernel: [    0.146553] acpiphp: Slot [7] registered
Oct 26 13:56:53 ctl01 systemd[1]: Starting Initial cloud-init job (metadata service crawler)...
Oct 26 13:56:53 ctl01 kernel: [    0.147431] acpiphp: Slot [9] registered
Oct 26 13:56:53 ctl01 kernel: [    0.148041] acpiphp: Slot [10] registered
Oct 26 13:56:53 ctl01 systemd[1]: Reached target Network.
Oct 26 13:56:53 ctl01 kernel: [    0.148877] acpiphp: Slot [11] registered
Oct 26 13:56:53 ctl01 kernel: [    0.149704] acpiphp: Slot [12] registered
Oct 26 13:56:53 ctl01 kernel: [    0.150535] acpiphp: Slot [13] registered
Oct 26 13:56:53 ctl01 kernel: [    0.151445] acpiphp: Slot [14] registered
Oct 26 13:56:53 ctl01 cloud-init[1867]: Cloud-init v. 19.2-36-g059d049c-0ubuntu2~16.04.1 running 'init' at Sat, 26 Oct 2019 13:56:52 +0000. Up 9.22 seconds.
Oct 26 13:56:53 ctl01 kernel: [    0.152040] acpiphp: Slot [15] registered
Oct 26 13:56:53 ctl01 kernel: [    0.152879] acpiphp: Slot [16] registered
Oct 26 13:56:53 ctl01 kernel: [    0.153716] acpiphp: Slot [17] registered
Oct 26 13:56:53 ctl01 kernel: [    0.154559] acpiphp: Slot [18] registered
Oct 26 13:56:53 ctl01 kernel: [    0.155486] acpiphp: Slot [19] registered
Oct 26 13:56:53 ctl01 kernel: [    0.156040] acpiphp: Slot [20] registered
Oct 26 13:56:53 ctl01 kernel: [    0.156873] acpiphp: Slot [21] registered
Oct 26 13:56:53 ctl01 kernel: [    0.157698] acpiphp: Slot [22] registered
Oct 26 13:56:53 ctl01 kernel: [    0.158550] acpiphp: Slot [23] registered
Oct 26 13:56:53 ctl01 kernel: [    0.159437] acpiphp: Slot [24] registered
Oct 26 13:56:53 ctl01 kernel: [    0.160045] acpiphp: Slot [25] registered
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: ++++++++++++++++++++++++++++++++++++++++Net device info++++++++++++++++++++++++++++++++++++++++
Oct 26 13:56:53 ctl01 kernel: [    0.160872] acpiphp: Slot [26] registered
Oct 26 13:56:53 ctl01 kernel: [    0.161703] acpiphp: Slot [27] registered
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: +-----------+-------+----------------------------+---------------+--------+-------------------+
Oct 26 13:56:53 ctl01 kernel: [    0.162595] acpiphp: Slot [28] registered
Oct 26 13:56:53 ctl01 kernel: [    0.163443] acpiphp: Slot [29] registered
Oct 26 13:56:53 ctl01 kernel: [    0.164040] acpiphp: Slot [30] registered
Oct 26 13:56:53 ctl01 kernel: [    0.164871] acpiphp: Slot [31] registered
Oct 26 13:56:53 ctl01 kernel: [    0.165679] PCI host bridge to bus 0000:00
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: |   Device  |   Up  |          Address           |      Mask     | Scope  |     Hw-Address    |
Oct 26 13:56:53 ctl01 kernel: [    0.166929] pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7 window]
Oct 26 13:56:53 ctl01 kernel: [    0.168004] pci_bus 0000:00: root bus resource [io  0x0d00-0xffff window]
Oct 26 13:56:53 ctl01 kernel: [    0.169256] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]
Oct 26 13:56:53 ctl01 kernel: [    0.170739] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]
Oct 26 13:56:53 ctl01 kernel: [    0.172004] pci_bus 0000:00: root bus resource [bus 00-ff]
Oct 26 13:56:53 ctl01 kernel: [    0.173081] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: +-----------+-------+----------------------------+---------------+--------+-------------------+
Oct 26 13:56:53 ctl01 kernel: [    0.173576] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100
Oct 26 13:56:53 ctl01 kernel: [    0.174295] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180
Oct 26 13:56:53 ctl01 kernel: [    0.179134] pci 0000:00:01.1: reg 0x20: [io  0xc140-0xc14f]
Oct 26 13:56:53 ctl01 kernel: [    0.181267] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io  0x01f0-0x01f7]
Oct 26 13:56:53 ctl01 kernel: [    0.182638] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io  0x03f6]
Oct 26 13:56:53 ctl01 kernel: [    0.183846] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io  0x0170-0x0177]
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: |  br-mgmt  |  True |        172.16.10.36        | 255.255.255.0 | global | 52:54:00:cc:bf:d7 |
Oct 26 13:56:53 ctl01 kernel: [    0.184004] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io  0x0376]
Oct 26 13:56:53 ctl01 kernel: [    0.185392] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000
Oct 26 13:56:53 ctl01 kernel: [    0.185875] pci 0000:00:01.3: quirk: [io  0x0600-0x063f] claimed by PIIX4 ACPI
Oct 26 13:56:53 ctl01 kernel: [    0.187253] pci 0000:00:01.3: quirk: [io  0x0700-0x070f] claimed by PIIX4 SMB
Oct 26 13:56:53 ctl01 kernel: [    0.188250] pci 0000:00:02.0: [1013:00b8] type 00 class 0x030000
Oct 26 13:56:53 ctl01 kernel: [    0.190841] pci 0000:00:02.0: reg 0x10: [mem 0xfc000000-0xfdffffff pref]
Oct 26 13:56:53 ctl01 kernel: [    0.193362] pci 0000:00:02.0: reg 0x14: [mem 0xfebd0000-0xfebd0fff]
Oct 26 13:56:53 ctl01 kernel: [    0.204007] pci 0000:00:02.0: reg 0x30: [mem 0xfebc0000-0xfebcffff pref]
Oct 26 13:56:53 ctl01 kernel: [    0.204241] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000
Oct 26 13:56:53 ctl01 kernel: [    0.205976] pci 0000:00:03.0: reg 0x10: [io  0xc040-0xc05f]
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: |  br-mgmt  |  True | fe80::5054:ff:fecc:bfd7/64 |       .       |  link  | 52:54:00:cc:bf:d7 |
Oct 26 13:56:53 ctl01 kernel: [    0.207591] pci 0000:00:03.0: reg 0x14: [mem 0xfebd1000-0xfebd1fff]
Oct 26 13:56:53 ctl01 kernel: [    0.216839] pci 0000:00:03.0: reg 0x30: [mem 0xfeac0000-0xfeafffff pref]
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: |    ens3   |  True |       192.168.11.21        | 255.255.255.0 | global | 52:54:00:d1:44:f2 |
Oct 26 13:56:53 ctl01 kernel: [    0.217200] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000
Oct 26 13:56:53 ctl01 kernel: [    0.219083] pci 0000:00:04.0: reg 0x10: [io  0xc060-0xc07f]
Oct 26 13:56:53 ctl01 kernel: [    0.220005] pci 0000:00:04.0: reg 0x14: [mem 0xfebd2000-0xfebd2fff]
Oct 26 13:56:53 ctl01 kernel: [    0.229790] pci 0000:00:04.0: reg 0x30: [mem 0xfeb00000-0xfeb3ffff pref]
Oct 26 13:56:53 ctl01 kernel: [    0.230184] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000
Oct 26 13:56:53 ctl01 kernel: [    0.232014] pci 0000:00:05.0: reg 0x10: [io  0xc080-0xc09f]
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: |    ens3   |  True | fe80::5054:ff:fed1:44f2/64 |       .       |  link  | 52:54:00:d1:44:f2 |
Oct 26 13:56:53 ctl01 kernel: [    0.233671] pci 0000:00:05.0: reg 0x14: [mem 0xfebd3000-0xfebd3fff]
Oct 26 13:56:53 ctl01 kernel: [    0.242481] pci 0000:00:05.0: reg 0x30: [mem 0xfeb40000-0xfeb7ffff pref]
Oct 26 13:56:53 ctl01 kernel: [    0.242846] pci 0000:00:06.0: [1af4:1000] type 00 class 0x020000
Oct 26 13:56:53 ctl01 kernel: [    0.244005] pci 0000:00:06.0: reg 0x10: [io  0xc0a0-0xc0bf]
Oct 26 13:56:53 ctl01 kernel: [    0.247009] pci 0000:00:06.0: reg 0x14: [mem 0xfebd4000-0xfebd4fff]
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: |    ens4   |  True | fe80::5054:ff:fecc:bfd7/64 |       .       |  link  | 52:54:00:cc:bf:d7 |
Oct 26 13:56:53 ctl01 kernel: [    0.254503] pci 0000:00:06.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref]
Oct 26 13:56:53 ctl01 kernel: [    0.254878] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000
Oct 26 13:56:53 ctl01 kernel: [    0.256927] pci 0000:00:07.0: reg 0x10: [io  0xc000-0xc03f]
Oct 26 13:56:53 ctl01 kernel: [    0.258781] pci 0000:00:07.0: reg 0x14: [mem 0xfebd5000-0xfebd5fff]
Oct 26 13:56:53 ctl01 kernel: [    0.268328] pci 0000:00:08.0: [8086:2934] type 00 class 0x0c0300
Oct 26 13:56:53 ctl01 kernel: [    0.272465] pci 0000:00:08.0: reg 0x20: [io  0xc0c0-0xc0df]
Oct 26 13:56:53 ctl01 kernel: [    0.274410] pci 0000:00:08.1: [8086:2935] type 00 class 0x0c0300
Oct 26 13:56:53 ctl01 kernel: [    0.280443] pci 0000:00:08.1: reg 0x20: [io  0xc0e0-0xc0ff]
Oct 26 13:56:53 ctl01 kernel: [    0.282415] pci 0000:00:08.2: [8086:2936] type 00 class 0x0c0300
Oct 26 13:56:53 ctl01 kernel: [    0.286897] pci 0000:00:08.2: reg 0x20: [io  0xc100-0xc11f]
Oct 26 13:56:53 ctl01 kernel: [    0.290079] pci 0000:00:08.7: [8086:293a] type 00 class 0x0c0320
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: |    ens5   |  True | fe80::5054:ff:fedd:4a72/64 |       .       |  link  | 52:54:00:dd:4a:72 |
Oct 26 13:56:53 ctl01 kernel: [    0.291070] pci 0000:00:08.7: reg 0x10: [mem 0xfebd6000-0xfebd6fff]
Oct 26 13:56:53 ctl01 kernel: [    0.296200] pci 0000:00:09.0: [1af4:1002] type 00 class 0x00ff00
Oct 26 13:56:53 ctl01 kernel: [    0.297149] pci 0000:00:09.0: reg 0x10: [io  0xc120-0xc13f]
Oct 26 13:56:53 ctl01 kernel: [    0.304686] ACPI: PCI Interrupt Link [LNKA] (IRQs 5 *10 11)
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: | ens5.1000 |  True | fe80::5054:ff:fedd:4a72/64 |       .       |  link  | 52:54:00:dd:4a:72 |
Oct 26 13:56:53 ctl01 kernel: [    0.305913] ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11)
Oct 26 13:56:53 ctl01 kernel: [    0.307135] ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11)
Oct 26 13:56:53 ctl01 kernel: [    0.308114] ACPI: PCI Interrupt Link [LNKD] (IRQs 5 10 *11)
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: |    ens6   | False |             .              |       .       |   .    | 52:54:00:23:fb:5b |
Oct 26 13:56:53 ctl01 kernel: [    0.309226] ACPI: PCI Interrupt Link [LNKS] (IRQs *9)
Oct 26 13:56:53 ctl01 kernel: [    0.311927] SCSI subsystem initialized
Oct 26 13:56:53 ctl01 kernel: [    0.312089] libata version 3.00 loaded.
Oct 26 13:56:53 ctl01 kernel: [    0.312089] pci 0000:00:02.0: vgaarb: setting as boot VGA device
Oct 26 13:56:53 ctl01 kernel: [    0.313214] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none
Oct 26 13:56:53 ctl01 kernel: [    0.314854] pci 0000:00:02.0: vgaarb: bridge control possible
Oct 26 13:56:53 ctl01 kernel: [    0.316006] vgaarb: loaded
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: |     lo    |  True |         127.0.0.1          |   255.0.0.0   |  host  |         .         |
Oct 26 13:56:53 ctl01 kernel: [    0.316612] ACPI: bus type USB registered
Oct 26 13:56:53 ctl01 kernel: [    0.317423] usbcore: registered new interface driver usbfs
Oct 26 13:56:53 ctl01 kernel: [    0.318514] usbcore: registered new interface driver hub
Oct 26 13:56:53 ctl01 kernel: [    0.319608] usbcore: registered new device driver usb
Oct 26 13:56:53 ctl01 kernel: [    0.320096] EDAC MC: Ver: 3.0.0
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: |     lo    |  True |          ::1/128           |       .       |  host  |         .         |
Oct 26 13:56:53 ctl01 kernel: [    0.321052] PCI: Using ACPI for IRQ routing
Oct 26 13:56:53 ctl01 kernel: [    0.321052] PCI: pci_cache_line_size set to 64 bytes
Oct 26 13:56:53 ctl01 kernel: [    0.321170] e820: reserve RAM buffer [mem 0x0009fc00-0x0009ffff]
Oct 26 13:56:53 ctl01 kernel: [    0.321171] e820: reserve RAM buffer [mem 0xbffdf000-0xbfffffff]
Oct 26 13:56:53 ctl01 kernel: [    0.321259] NetLabel: Initializing
Oct 26 13:56:53 ctl01 kernel: [    0.321959] NetLabel:  domain hash size = 128
Oct 26 13:56:53 ctl01 kernel: [    0.324004] NetLabel:  protocols = UNLABELED CIPSOv4 CALIPSO
Oct 26 13:56:53 ctl01 kernel: [    0.325096] NetLabel:  unlabeled traffic allowed by default
Oct 26 13:56:53 ctl01 kernel: [    0.326187] clocksource: Switched to clocksource kvm-clock
Oct 26 13:56:53 ctl01 kernel: [    0.335852] VFS: Disk quotas dquot_6.6.0
Oct 26 13:56:53 ctl01 kernel: [    0.336746] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: +-----------+-------+----------------------------+---------------+--------+-------------------+
Oct 26 13:56:53 ctl01 kernel: [    0.338102] AppArmor: AppArmor Filesystem Enabled
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: ++++++++++++++++++++++++++++++Route IPv4 info++++++++++++++++++++++++++++++
Oct 26 13:56:53 ctl01 kernel: [    0.339027] pnp: PnP ACPI init
Oct 26 13:56:53 ctl01 kernel: [    0.339720] pnp 00:00: Plug and Play ACPI device, IDs PNP0b00 (active)
Oct 26 13:56:53 ctl01 kernel: [    0.339761] pnp 00:01: Plug and Play ACPI device, IDs PNP0303 (active)
Oct 26 13:56:53 ctl01 kernel: [    0.339791] pnp 00:02: Plug and Play ACPI device, IDs PNP0f13 (active)
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: +-------+--------------+--------------+---------------+-----------+-------+
Oct 26 13:56:53 ctl01 kernel: [    0.339821] pnp 00:03: [dma 2]
Oct 26 13:56:53 ctl01 kernel: [    0.339831] pnp 00:03: Plug and Play ACPI device, IDs PNP0700 (active)
Oct 26 13:56:53 ctl01 kernel: [    0.339912] pnp 00:04: Plug and Play ACPI device, IDs PNP0501 (active)
Oct 26 13:56:53 ctl01 kernel: [    0.340159] pnp: PnP ACPI: found 5 devices
Oct 26 13:56:53 ctl01 kernel: [    0.347610] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
Oct 26 13:56:53 ctl01 kernel: [    0.349269] pci_bus 0000:00: resource 4 [io  0x0000-0x0cf7 window]
Oct 26 13:56:53 ctl01 kernel: [    0.349270] pci_bus 0000:00: resource 5 [io  0x0d00-0xffff window]
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: | Route | Destination  |   Gateway    |    Genmask    | Interface | Flags |
Oct 26 13:56:53 ctl01 kernel: [    0.349271] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]
Oct 26 13:56:53 ctl01 kernel: [    0.349272] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window]
Oct 26 13:56:53 ctl01 kernel: [    0.349326] NET: Registered protocol family 2
Oct 26 13:56:53 ctl01 kernel: [    0.350407] TCP established hash table entries: 131072 (order: 8, 1048576 bytes)
Oct 26 13:56:53 ctl01 kernel: [    0.351961] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: +-------+--------------+--------------+---------------+-----------+-------+
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: |   0   |   0.0.0.0    | 192.168.11.3 |    0.0.0.0    |    ens3   |   UG  |
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: |   1   |  10.254.0.0  | 172.16.10.36 |  255.255.0.0  |  br-mgmt  |   UG  |
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: |   2   | 172.16.10.0  |   0.0.0.0    | 255.255.255.0 |  br-mgmt  |   U   |
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: |   3   | 192.168.11.0 |   0.0.0.0    | 255.255.255.0 |    ens3   |   U   |
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: +-------+--------------+--------------+---------------+-----------+-------+
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: +++++++++++++++++++Route IPv6 info+++++++++++++++++++
Oct 26 13:56:53 ctl01 kernel: [    0.353352] TCP: Hash tables configured (established 131072 bind 65536)
Oct 26 13:56:53 ctl01 kernel: [    0.354636] UDP hash table entries: 8192 (order: 6, 262144 bytes)
Oct 26 13:56:53 ctl01 kernel: [    0.355808] UDP-Lite hash table entries: 8192 (order: 6, 262144 bytes)
Oct 26 13:56:53 ctl01 kernel: [    0.357103] NET: Registered protocol family 1
Oct 26 13:56:53 ctl01 kernel: [    0.357967] pci 0000:00:00.0: Limiting direct PCI/PCI transfers
Oct 26 13:56:53 ctl01 kernel: [    0.359084] pci 0000:00:01.0: PIIX3: Enabling Passive Release
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: +-------+-------------+---------+-----------+-------+
Oct 26 13:56:53 ctl01 kernel: [    0.360220] pci 0000:00:01.0: Activating ISA DMA hang workarounds
Oct 26 13:56:53 ctl01 kernel: [    0.361417] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]
Oct 26 13:56:53 ctl01 kernel: [    0.385957] ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 11
Oct 26 13:56:53 ctl01 kernel: [    0.434413] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 10
Oct 26 13:56:53 ctl01 kernel: [    0.481897] ACPI: PCI Interrupt Link [LNKB] enabled at IRQ 10
Oct 26 13:56:53 ctl01 kernel: [    0.530031] ACPI: PCI Interrupt Link [LNKC] enabled at IRQ 11
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: | Route | Destination | Gateway | Interface | Flags |
Oct 26 13:56:53 ctl01 kernel: [    0.555112] PCI: CLS 0 bytes, default 64
Oct 26 13:56:53 ctl01 kernel: [    0.555161] Unpacking initramfs...
Oct 26 13:56:53 ctl01 kernel: [    1.159398] Freeing initrd memory: 53652K
Oct 26 13:56:53 ctl01 kernel: [    1.161969] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
Oct 26 13:56:53 ctl01 kernel: [    1.163319] software IO TLB: mapped [mem 0xbbfdf000-0xbffdf000] (64MB)
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: +-------+-------------+---------+-----------+-------+
Oct 26 13:56:53 ctl01 kernel: [    1.164751] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x285c3aeaff3, max_idle_ns: 440795255742 ns
Oct 26 13:56:53 ctl01 kernel: [    1.166860] Scanning for low memory corruption every 60 seconds
Oct 26 13:56:53 ctl01 kernel: [    1.169272] Initialise system trusted keyrings
Oct 26 13:56:53 ctl01 kernel: [    1.170396] Key type blacklist registered
Oct 26 13:56:53 ctl01 kernel: [    1.171424] workingset: timestamp_bits=36 max_order=22 bucket_order=0
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: |   0   |  fe80::/64  |    ::   |    ens4   |   U   |
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: |   1   |  fe80::/64  |    ::   |    ens3   |   U   |
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: |   2   |  fe80::/64  |    ::   |  br-mgmt  |   U   |
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: |   3   |  fe80::/64  |    ::   |    ens5   |   U   |
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: |   4   |  fe80::/64  |    ::   | ens5.1000 |   U   |
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: |   6   |    local    |    ::   |    ens4   |   U   |
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: |   7   |   ff00::/8  |    ::   |    ens4   |   U   |
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: |   8   |   ff00::/8  |    ::   |    ens3   |   U   |
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: |   9   |   ff00::/8  |    ::   |  br-mgmt  |   U   |
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: |   10  |   ff00::/8  |    ::   |    ens5   |   U   |
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: |   11  |   ff00::/8  |    ::   | ens5.1000 |   U   |
Oct 26 13:56:53 ctl01 cloud-init[1867]: ci-info: +-------+-------------+---------+-----------+-------+
Oct 26 13:56:53 ctl01 systemd[1]: Started Initial cloud-init job (metadata service crawler).
Oct 26 13:56:53 ctl01 systemd[1]: Reached target Network is Online.
Oct 26 13:56:53 ctl01 systemd[1]: Starting iSCSI initiator daemon (iscsid)...
Oct 26 13:56:53 ctl01 systemd[1]: Reached target System Initialization.
Oct 26 13:56:53 ctl01 systemd[1]: Listening on D-Bus System Message Bus Socket.
Oct 26 13:56:53 ctl01 systemd[1]: Starting LXD - unix socket.
Oct 26 13:56:53 ctl01 systemd[1]: Started Daily Cleanup of Temporary Directories.
Oct 26 13:56:53 ctl01 systemd[1]: Started Daily apt download activities.
Oct 26 13:56:53 ctl01 systemd[1]: Starting Socket activation for snappy daemon.
Oct 26 13:56:53 ctl01 systemd[1]: Listening on ACPID Listen Socket.
Oct 26 13:56:53 ctl01 systemd[1]: Started Daily apt upgrade and clean activities.
Oct 26 13:56:53 ctl01 systemd[1]: Started Message of the Day.
Oct 26 13:56:53 ctl01 systemd[1]: Reached target Timers.
Oct 26 13:56:53 ctl01 systemd[1]: Listening on UUID daemon activation socket.
Oct 26 13:56:53 ctl01 systemd[1]: Started ACPI Events Check.
Oct 26 13:56:53 ctl01 systemd[1]: Reached target Paths.
Oct 26 13:56:53 ctl01 systemd[1]: Reached target Cloud-config availability.
Oct 26 13:56:53 ctl01 systemd[1]: Listening on LXD - unix socket.
Oct 26 13:56:53 ctl01 systemd[1]: Listening on Socket activation for snappy daemon.
Oct 26 13:56:53 ctl01 kernel: [    1.174143] zbud: loaded
Oct 26 13:56:53 ctl01 iscsid: iSCSI logger with pid=1953 started!
Oct 26 13:56:53 ctl01 kernel: [    1.175338] squashfs: version 4.0 (2009/01/31) Phillip Lougher
Oct 26 13:56:53 ctl01 kernel: [    1.176829] fuse init (API version 7.26)
Oct 26 13:56:53 ctl01 kernel: [    1.179623] Key type asymmetric registered
Oct 26 13:56:53 ctl01 kernel: [    1.180697] Asymmetric key parser 'x509' registered
Oct 26 13:56:53 ctl01 kernel: [    1.181783] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 246)
Oct 26 13:56:53 ctl01 kernel: [    1.183438] io scheduler noop registered
Oct 26 13:56:53 ctl01 kernel: [    1.184332] io scheduler deadline registered
Oct 26 13:56:53 ctl01 systemd[1]: Started iSCSI initiator daemon (iscsid).
Oct 26 13:56:53 ctl01 kernel: [    1.185314] io scheduler cfq registered (default)
Oct 26 13:56:53 ctl01 kernel: [    1.186826] intel_idle: Please enable MWAIT in BIOS SETUP
Oct 26 13:56:53 ctl01 kernel: [    1.186893] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
Oct 26 13:56:53 ctl01 kernel: [    1.188596] ACPI: Power Button [PWRF]
Oct 26 13:56:53 ctl01 kernel: [    1.215613] virtio-pci 0000:00:03.0: virtio_pci: leaving for legacy driver
Oct 26 13:56:53 ctl01 systemd[1]: Starting Login to default iSCSI targets...
Oct 26 13:56:53 ctl01 kernel: [    1.243289] virtio-pci 0000:00:04.0: virtio_pci: leaving for legacy driver
Oct 26 13:56:53 ctl01 kernel: [    1.271276] virtio-pci 0000:00:05.0: virtio_pci: leaving for legacy driver
Oct 26 13:56:53 ctl01 kernel: [    1.299447] virtio-pci 0000:00:06.0: virtio_pci: leaving for legacy driver
Oct 26 13:56:53 ctl01 kernel: [    1.327193] virtio-pci 0000:00:07.0: virtio_pci: leaving for legacy driver
Oct 26 13:56:53 ctl01 kernel: [    1.354456] virtio-pci 0000:00:09.0: virtio_pci: leaving for legacy driver
Oct 26 13:56:53 ctl01 kernel: [    1.356986] Serial: 8250/16550 driver, 32 ports, IRQ sharing enabled
Oct 26 13:56:53 ctl01 systemd[1]: Reached target Sockets.
Oct 26 13:56:53 ctl01 kernel: [    1.384996] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
Oct 26 13:56:53 ctl01 kernel: [    1.388998] Linux agpgart interface v0.103
Oct 26 13:56:53 ctl01 kernel: [    1.392695] loop: module loaded
Oct 26 13:56:53 ctl01 kernel: [    1.393535] ata_piix 0000:00:01.1: version 2.13
Oct 26 13:56:53 ctl01 kernel: [    1.395071] scsi host0: ata_piix
Oct 26 13:56:53 ctl01 kernel: [    1.396098] scsi host1: ata_piix
Oct 26 13:56:53 ctl01 systemd[1]: Reached target Basic System.
Oct 26 13:56:53 ctl01 kernel: [    1.396984] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc140 irq 14
Oct 26 13:56:53 ctl01 kernel: [    1.398547] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc148 irq 15
Oct 26 13:56:53 ctl01 kernel: [    1.400211] libphy: Fixed MDIO Bus: probed
Oct 26 13:56:53 ctl01 kernel: [    1.401355] tun: Universal TUN/TAP device driver, 1.6
Oct 26 13:56:53 ctl01 kernel: [    1.402655] PPP generic driver version 2.4.2
Oct 26 13:56:53 ctl01 systemd[1]: Starting The Salt Minion...
Oct 26 13:56:53 ctl01 kernel: [    1.403808] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
Oct 26 13:56:53 ctl01 kernel: [    1.405194] ehci-pci: EHCI PCI platform driver
Oct 26 13:56:53 ctl01 kernel: [    1.429864] ehci-pci 0000:00:08.7: EHCI Host Controller
Oct 26 13:56:53 ctl01 kernel: [    1.437670] ehci-pci 0000:00:08.7: new USB bus registered, assigned bus number 1
Oct 26 13:56:53 ctl01 kernel: [    1.439251] ehci-pci 0000:00:08.7: irq 11, io mem 0xfebd6000
Oct 26 13:56:53 ctl01 kernel: [    1.456062] ehci-pci 0000:00:08.7: USB 2.0 started, EHCI 1.00
Oct 26 13:56:53 ctl01 kernel: [    1.457505] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
Oct 26 13:56:53 ctl01 kernel: [    1.459061] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
Oct 26 13:56:53 ctl01 kernel: [    1.460776] usb usb1: Product: EHCI Host Controller
Oct 26 13:56:53 ctl01 kernel: [    1.461953] usb usb1: Manufacturer: Linux 4.15.0-66-generic ehci_hcd
Oct 26 13:56:53 ctl01 systemd[1]: Starting Thermal Daemon Service...
Oct 26 13:56:53 ctl01 kernel: [    1.463418] usb usb1: SerialNumber: 0000:00:08.7
Oct 26 13:56:53 ctl01 kernel: [    1.464714] hub 1-0:1.0: USB hub found
Oct 26 13:56:53 ctl01 kernel: [    1.465668] hub 1-0:1.0: 6 ports detected
Oct 26 13:56:53 ctl01 kernel: [    1.466889] ehci-platform: EHCI generic platform driver
Oct 26 13:56:53 ctl01 systemd[1]: Starting LSB: Set sysfs variables from /etc/sysfs.conf...
Oct 26 13:56:53 ctl01 kernel: [    1.468154] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
Oct 26 13:56:53 ctl01 kernel: [    1.469428] ohci-pci: OHCI PCI platform driver
Oct 26 13:56:53 ctl01 kernel: [    1.470298] ohci-platform: OHCI generic platform driver
Oct 26 13:56:53 ctl01 kernel: [    1.471294] uhci_hcd: USB Universal Host Controller Interface driver
Oct 26 13:56:53 ctl01 kernel: [    1.496178] uhci_hcd 0000:00:08.0: UHCI Host Controller
Oct 26 13:56:53 ctl01 systemd[1]: Starting /etc/rc.local Compatibility...
Oct 26 13:56:53 ctl01 kernel: [    1.497219] uhci_hcd 0000:00:08.0: new USB bus registered, assigned bus number 2
Oct 26 13:56:53 ctl01 kernel: [    1.498632] uhci_hcd 0000:00:08.0: detected 2 ports
Oct 26 13:56:53 ctl01 kernel: [    1.499633] uhci_hcd 0000:00:08.0: irq 11, io base 0x0000c0c0
Oct 26 13:56:53 ctl01 kernel: [    1.500835] usb usb2: New USB device found, idVendor=1d6b, idProduct=0001
Oct 26 13:56:53 ctl01 kernel: [    1.502154] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
Oct 26 13:56:53 ctl01 systemd[1]: Started Deferred execution scheduler.
Oct 26 13:56:53 ctl01 kernel: [    1.503575] usb usb2: Product: UHCI Host Controller
Oct 26 13:56:53 ctl01 kernel: [    1.504626] usb usb2: Manufacturer: Linux 4.15.0-66-generic uhci_hcd
Oct 26 13:56:53 ctl01 kernel: [    1.505894] usb usb2: SerialNumber: 0000:00:08.0
Oct 26 13:56:53 ctl01 kernel: [    1.506974] hub 2-0:1.0: USB hub found
Oct 26 13:56:53 ctl01 kernel: [    1.507759] hub 2-0:1.0: 2 ports detected
Oct 26 13:56:53 ctl01 systemd[1]: Started ACPI event daemon.
Oct 26 13:56:53 ctl01 kernel: [    1.532945] uhci_hcd 0000:00:08.1: UHCI Host Controller
Oct 26 13:56:53 ctl01 kernel: [    1.533975] uhci_hcd 0000:00:08.1: new USB bus registered, assigned bus number 3
Oct 26 13:56:53 ctl01 kernel: [    1.535580] uhci_hcd 0000:00:08.1: detected 2 ports
Oct 26 13:56:53 ctl01 kernel: [    1.536674] uhci_hcd 0000:00:08.1: irq 10, io base 0x0000c0e0
Oct 26 13:56:53 ctl01 kernel: [    1.537878] usb usb3: New USB device found, idVendor=1d6b, idProduct=0001
Oct 26 13:56:53 ctl01 kernel: [    1.539201] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
Oct 26 13:56:53 ctl01 kernel: [    1.540675] usb usb3: Product: UHCI Host Controller
Oct 26 13:56:53 ctl01 kernel: [    1.541703] usb usb3: Manufacturer: Linux 4.15.0-66-generic uhci_hcd
Oct 26 13:56:53 ctl01 kernel: [    1.543087] usb usb3: SerialNumber: 0000:00:08.1
Oct 26 13:56:53 ctl01 kernel: [    1.544150] hub 3-0:1.0: USB hub found
Oct 26 13:56:53 ctl01 kernel: [    1.544972] hub 3-0:1.0: 2 ports detected
Oct 26 13:56:53 ctl01 kernel: [    1.560728] ata1.01: NODEV after polling detection
Oct 26 13:56:53 ctl01 systemd[1]: Starting Login Service...
Oct 26 13:56:53 ctl01 kernel: [    1.561059] ata1.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100
Oct 26 13:56:53 ctl01 kernel: [    1.562982] ata1.00: configured for MWDMA2
Oct 26 13:56:53 ctl01 kernel: [    1.564764] scsi 0:0:0:0: CD-ROM            QEMU     QEMU DVD-ROM     2.5+ PQ: 0 ANSI: 5
Oct 26 13:56:53 ctl01 kernel: [    1.567372] sr 0:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray
Oct 26 13:56:53 ctl01 kernel: [    1.568733] cdrom: Uniform CD-ROM driver Revision: 3.20
Oct 26 13:56:53 ctl01 kernel: [    1.569984] sr 0:0:0:0: Attached scsi CD-ROM sr0
Oct 26 13:56:53 ctl01 systemd[1]: Started FUSE filesystem for LXC.
Oct 26 13:56:53 ctl01 kernel: [    1.570021] sr 0:0:0:0: Attached scsi generic sg0 type 5
Oct 26 13:56:53 ctl01 kernel: [    1.574592] uhci_hcd 0000:00:08.2: UHCI Host Controller
Oct 26 13:56:53 ctl01 kernel: [    1.575699] uhci_hcd 0000:00:08.2: new USB bus registered, assigned bus number 4
Oct 26 13:56:53 ctl01 kernel: [    1.577231] uhci_hcd 0000:00:08.2: detected 2 ports
Oct 26 13:56:53 ctl01 kernel: [    1.578304] uhci_hcd 0000:00:08.2: irq 10, io base 0x0000c100
Oct 26 13:56:53 ctl01 kernel: [    1.579554] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001
Oct 26 13:56:53 ctl01 systemd[1]: Starting Open vSwitch Internal Unit...
Oct 26 13:56:53 ctl01 kernel: [    1.580930] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
Oct 26 13:56:53 ctl01 kernel: [    1.582411] usb usb4: Product: UHCI Host Controller
Oct 26 13:56:53 ctl01 kernel: [    1.583521] usb usb4: Manufacturer: Linux 4.15.0-66-generic uhci_hcd
Oct 26 13:56:53 ctl01 kernel: [    1.584843] usb usb4: SerialNumber: 0000:00:08.2
Oct 26 13:56:53 ctl01 kernel: [    1.585897] hub 4-0:1.0: USB hub found
Oct 26 13:56:53 ctl01 kernel: [    1.586718] hub 4-0:1.0: 2 ports detected
Oct 26 13:56:53 ctl01 systemd[1]: Starting System Logging Service...
Oct 26 13:56:53 ctl01 kernel: [    1.587697] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12
Oct 26 13:56:53 ctl01 kernel: [    1.591044] serio: i8042 KBD port at 0x60,0x64 irq 1
Oct 26 13:56:53 ctl01 kernel: [    1.592059] serio: i8042 AUX port at 0x60,0x64 irq 12
Oct 26 13:56:53 ctl01 kernel: [    1.593204] mousedev: PS/2 mouse device common for all mice
Oct 26 13:56:53 ctl01 kernel: [    1.594594] rtc_cmos 00:00: RTC can wake from S4
Oct 26 13:56:53 ctl01 kernel: [    1.596016] rtc_cmos 00:00: rtc core: registered rtc_cmos as rtc0
Oct 26 13:56:53 ctl01 systemd[1]: Starting LSB: MD monitoring daemon...
Oct 26 13:56:53 ctl01 kernel: [    1.597264] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1
Oct 26 13:56:53 ctl01 kernel: [    1.598995] rtc_cmos 00:00: alarms up to one day, 114 bytes nvram
Oct 26 13:56:53 ctl01 kernel: [    1.600326] i2c /dev entries driver
Oct 26 13:56:53 ctl01 kernel: [    1.601094] device-mapper: uevent: version 1.0.3
Oct 26 13:56:53 ctl01 kernel: [    1.602113] device-mapper: ioctl: 4.37.0-ioctl (2017-09-20) initialised: dm-devel@redhat.com
Oct 26 13:56:53 ctl01 systemd[1]: Starting LSB: Record successful boot for GRUB...
Oct 26 13:56:53 ctl01 systemd[1]: Starting LXD - container startup/shutdown...
Oct 26 13:56:53 ctl01 systemd[1]: Started Regular background program processing daemon.
Oct 26 13:56:53 ctl01 cron[2044]: (CRON) INFO (pidfile fd = 3)
Oct 26 13:56:53 ctl01 systemd[1]: Started Self Monitoring and Reporting Technology (SMART) Daemon.
Oct 26 13:56:53 ctl01 kernel: [    1.604147] ledtrig-cpu: registered to indicate activity on CPUs
Oct 26 13:56:53 ctl01 kernel: [    1.605922] NET: Registered protocol family 10
Oct 26 13:56:53 ctl01 kernel: [    1.611295] Segment Routing with IPv6
Oct 26 13:56:53 ctl01 lxcfs[2001]: hierarchies:
Oct 26 13:56:53 ctl01 kernel: [    1.612079] NET: Registered protocol family 17
Oct 26 13:56:53 ctl01 kernel: [    1.613128] Key type dns_resolver registered
Oct 26 13:56:53 ctl01 kernel: [    1.614874] mce: Using 10 MCE banks
Oct 26 13:56:53 ctl01 kernel: [    1.615633] RAS: Correctable Errors collector initialized.
Oct 26 13:56:53 ctl01 kernel: [    1.616764] sched_clock: Marking stable (1616732347, 0)->(2076965551, -460233204)
Oct 26 13:56:53 ctl01 lxcfs[2001]:   0: fd:   5: perf_event
Oct 26 13:56:53 ctl01 kernel: [    1.618656] registered taskstats version 1
Oct 26 13:56:53 ctl01 kernel: [    1.619516] Loading compiled-in X.509 certificates
Oct 26 13:56:53 ctl01 kernel: [    1.623156] Loaded X.509 cert 'Build time autogenerated kernel key: b403aa2f6b5a8b809b81566991891c544ad7d3fe'
Oct 26 13:56:53 ctl01 kernel: [    1.625174] zswap: loaded using pool lzo/zbud
Oct 26 13:56:53 ctl01 kernel: [    1.630298] Key type big_key registered
Oct 26 13:56:53 ctl01 kernel: [    1.631119] Key type trusted registered
Oct 26 13:56:53 ctl01 kernel: [    1.634377] Key type encrypted registered
Oct 26 13:56:53 ctl01 kernel: [    1.635257] AppArmor: AppArmor sha1 policy hashing enabled
Oct 26 13:56:53 ctl01 lxcfs[2001]:   1: fd:   6: cpuset
Oct 26 13:56:53 ctl01 kernel: [    1.636414] ima: No TPM chip found, activating TPM-bypass! (rc=-19)
Oct 26 13:56:53 ctl01 kernel: [    1.637613] ima: Allocated hash algorithm: sha1
Oct 26 13:56:53 ctl01 kernel: [    1.638524] evm: HMAC attrs: 0x1
Oct 26 13:56:53 ctl01 kernel: [    1.639579]   Magic number: 7:249:939
Oct 26 13:56:53 ctl01 lxcfs[2001]:   2: fd:   7: freezer
Oct 26 13:56:53 ctl01 kernel: [    1.640600] rtc_cmos 00:00: setting system clock to 2019-10-26 13:56:44 UTC (1572098204)
Oct 26 13:56:53 ctl01 kernel: [    1.642347] BIOS EDD facility v0.16 2004-Jun-25, 0 devices found
Oct 26 13:56:53 ctl01 kernel: [    1.643530] EDD information not available.
Oct 26 13:56:53 ctl01 kernel: [    1.647126] Freeing unused kernel image memory: 2428K
Oct 26 13:56:53 ctl01 kernel: [    1.676074] Write protecting the kernel read-only data: 20480k
Oct 26 13:56:53 ctl01 kernel: [    1.678107] Freeing unused kernel image memory: 2008K
Oct 26 13:56:53 ctl01 lxcfs[2001]:   3: fd:   8: memory
Oct 26 13:56:53 ctl01 kernel: [    1.679754] Freeing unused kernel image memory: 1936K
Oct 26 13:56:53 ctl01 kernel: [    1.687229] x86/mm: Checked W+X mappings: passed, no W+X pages found.
Oct 26 13:56:53 ctl01 kernel: [    1.688605] x86/mm: Checking user space page tables
Oct 26 13:56:53 ctl01 kernel: [    1.695916] x86/mm: Checked W+X mappings: passed, no W+X pages found.
Oct 26 13:56:53 ctl01 kernel: [    1.712917] random: systemd-udevd: uninitialized urandom read (16 bytes read)
Oct 26 13:56:53 ctl01 kernel: [    1.713756] random: udevadm: uninitialized urandom read (16 bytes read)
Oct 26 13:56:53 ctl01 kernel: [    1.715870] random: systemd-udevd: uninitialized urandom read (16 bytes read)
Oct 26 13:56:53 ctl01 kernel: [    1.779685]  vda: vda1
Oct 26 13:56:53 ctl01 lxcfs[2001]:   4: fd:   9: rdma
Oct 26 13:56:53 ctl01 kernel: [    1.786618] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4
Oct 26 13:56:53 ctl01 kernel: [    1.788982] FDC 0 is a S82078B
Oct 26 13:56:53 ctl01 kernel: [    1.790218] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3
Oct 26 13:56:53 ctl01 kernel: [    1.795589] AVX version of gcm_enc/dec engaged.
Oct 26 13:56:53 ctl01 kernel: [    1.796570] AES CTR mode by8 optimization enabled
Oct 26 13:56:53 ctl01 lxcfs[2001]:   5: fd:  10: devices
Oct 26 13:56:53 ctl01 kernel: [    1.805058] virtio_net virtio0 ens3: renamed from eth0
Oct 26 13:56:53 ctl01 kernel: [    1.807980] [TTM] Zone  kernel: Available graphics memory: 7180184 kiB
Oct 26 13:56:53 ctl01 kernel: [    1.810013] [TTM] Zone   dma32: Available graphics memory: 2097152 kiB
Oct 26 13:56:53 ctl01 kernel: [    1.811763] [TTM] Initializing pool allocator
Oct 26 13:56:53 ctl01 lxcfs[2001]:   6: fd:  11: blkio
Oct 26 13:56:53 ctl01 kernel: [    1.813546] [TTM] Initializing DMA pool allocator
Oct 26 13:56:53 ctl01 kernel: [    1.815460] [drm] fb mappable at 0xFC000000
Oct 26 13:56:53 ctl01 kernel: [    1.816817] [drm] vram aper at 0xFC000000
Oct 26 13:56:53 ctl01 kernel: [    1.817913] [drm] size 33554432
Oct 26 13:56:53 ctl01 kernel: [    1.818736] [drm] fb depth is 24
Oct 26 13:56:53 ctl01 kernel: [    1.819578] [drm]    pitch is 3072
Oct 26 13:56:53 ctl01 kernel: [    1.820629] fbcon: cirrusdrmfb (fb0) is primary device
Oct 26 13:56:53 ctl01 kernel: [    1.828703] virtio_net virtio3 ens6: renamed from eth3
Oct 26 13:56:53 ctl01 kernel: [    1.843580] Console: switching to colour frame buffer device 128x48
Oct 26 13:56:53 ctl01 lxcfs[2001]:   7: fd:  12: pids
Oct 26 13:56:53 ctl01 kernel: [    1.849022] cirrus 0000:00:02.0: fb0: cirrusdrmfb frame buffer device
Oct 26 13:56:53 ctl01 kernel: [    1.856148] virtio_net virtio2 ens5: renamed from eth2
Oct 26 13:56:53 ctl01 kernel: [    1.868052] [drm] Initialized cirrus 1.0.0 20110418 for 0000:00:02.0 on minor 0
Oct 26 13:56:53 ctl01 kernel: [    1.884182] virtio_net virtio1 ens4: renamed from eth1
Oct 26 13:56:53 ctl01 kernel: [    2.044035] raid6: sse2x1   gen()  5711 MB/s
Oct 26 13:56:53 ctl01 kernel: [    2.092041] raid6: sse2x1   xor()  6986 MB/s
Oct 26 13:56:53 ctl01 lxcfs[2001]:   8: fd:  13: cpu,cpuacct
Oct 26 13:56:53 ctl01 kernel: [    2.140030] raid6: sse2x2   gen()  7753 MB/s
Oct 26 13:56:53 ctl01 kernel: [    2.188029] raid6: sse2x2   xor()  6456 MB/s
Oct 26 13:56:53 ctl01 kernel: [    2.236035] raid6: sse2x4   gen() 10169 MB/s
Oct 26 13:56:53 ctl01 kernel: [    2.284024] raid6: sse2x4   xor()  7672 MB/s
Oct 26 13:56:53 ctl01 kernel: [    2.284818] raid6: using algorithm sse2x4 gen() 10169 MB/s
Oct 26 13:56:53 ctl01 kernel: [    2.285802] raid6: .... xor() 7672 MB/s, rmw enabled
Oct 26 13:56:53 ctl01 lxcfs[2001]:   9: fd:  14: net_cls,net_prio
Oct 26 13:56:53 ctl01 kernel: [    2.286697] raid6: using ssse3x2 recovery algorithm
Oct 26 13:56:53 ctl01 kernel: [    2.289675] xor: automatically using best checksumming function   avx       
Oct 26 13:56:53 ctl01 kernel: [    2.293205] async_tx: api initialized (async)
Oct 26 13:56:53 ctl01 kernel: [    2.348314] Btrfs loaded, crc32c=crc32c-intel
Oct 26 13:56:53 ctl01 kernel: [    2.397285] EXT4-fs (vda1): mounted filesystem with ordered data mode. Opts: (null)
Oct 26 13:56:53 ctl01 kernel: [    2.550925] random: fast init done
Oct 26 13:56:53 ctl01 lxcfs[2001]:  10: fd:  15: hugetlb
Oct 26 13:56:53 ctl01 kernel: [    3.426907] Loading iSCSI transport class v2.0-870.
Oct 26 13:56:53 ctl01 kernel: [    3.428078] EXT4-fs (vda1): re-mounted. Opts: (null)
Oct 26 13:56:53 ctl01 kernel: [    3.449499] iscsi: registered transport (tcp)
Oct 26 13:56:53 ctl01 kernel: [    3.489380] iscsi: registered transport (iser)
Oct 26 13:56:53 ctl01 kernel: [    3.617569] nf_conntrack version 0.5.0 (65536 buckets, 262144 max)
Oct 26 13:56:53 ctl01 lxcfs[2001]:  11: fd:  16: name=systemd
Oct 26 13:56:53 ctl01 kernel: [    4.465067] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0
Oct 26 13:56:53 ctl01 kernel: [    4.875380] bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this.
Oct 26 13:56:53 ctl01 kernel: [    4.880883] audit: type=1400 audit(1572098207.732:2): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/bin/lxc-start" pid=946 comm="apparmor_parser"
Oct 26 13:56:53 ctl01 kernel: [    4.881685] br-mgmt: port 1(ens4) entered blocking state
Oct 26 13:56:53 ctl01 kernel: [    4.881687] br-mgmt: port 1(ens4) entered disabled state
Oct 26 13:56:53 ctl01 kernel: [    4.881765] device ens4 entered promiscuous mode
Oct 26 13:56:53 ctl01 cron[2044]: (CRON) INFO (Running @reboot jobs)
Oct 26 13:56:53 ctl01 kernel: [    4.902505] audit: type=1400 audit(1572098207.756:3): apparmor="STATUS" operation="profile_load" profile="unconfined" name="lxc-container-default" pid=944 comm="apparmor_parser"
Oct 26 13:56:53 ctl01 kernel: [    4.902509] audit: type=1400 audit(1572098207.756:4): apparmor="STATUS" operation="profile_load" profile="unconfined" name="lxc-container-default-cgns" pid=944 comm="apparmor_parser"
Oct 26 13:56:53 ctl01 kernel: [    4.902511] audit: type=1400 audit(1572098207.756:5): apparmor="STATUS" operation="profile_load" profile="unconfined" name="lxc-container-default-with-mounting" pid=944 comm="apparmor_parser"
Oct 26 13:56:53 ctl01 kernel: [    4.902514] audit: type=1400 audit(1572098207.756:6): apparmor="STATUS" operation="profile_load" profile="unconfined" name="lxc-container-default-with-nesting" pid=944 comm="apparmor_parser"
Oct 26 13:56:53 ctl01 kernel: [    4.903015] audit: type=1400 audit(1572098207.756:7): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/lib/snapd/snap-confine" pid=949 comm="apparmor_parser"
Oct 26 13:56:53 ctl01 kernel: [    4.903018] audit: type=1400 audit(1572098207.756:8): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/lib/snapd/snap-confine//mount-namespace-capture-helper" pid=949 comm="apparmor_parser"
Oct 26 13:56:53 ctl01 kernel: [    4.908917] audit: type=1400 audit(1572098207.764:9): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/lib/lxd/lxd-bridge-proxy" pid=948 comm="apparmor_parser"
Oct 26 13:56:53 ctl01 kernel: [    4.913228] audit: type=1400 audit(1572098207.768:10): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/sbin/tcpdump" pid=951 comm="apparmor_parser"
Oct 26 13:56:53 ctl01 kernel: [    4.920456] EDAC sbridge: Seeking for: PCI ID 8086:0ea0
Oct 26 13:56:53 ctl01 kernel: [    4.920462] EDAC sbridge:  Ver: 1.1.2 
Oct 26 13:56:53 ctl01 kernel: [    5.119818] audit: type=1400 audit(1572098207.976:11): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/sbin/dhclient" pid=945 comm="apparmor_parser"
Oct 26 13:56:53 ctl01 acpid: starting up with netlink and the input layer
Oct 26 13:56:53 ctl01 kernel: [    5.119822] audit: type=1400 audit(1572098207.976:12): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/lib/NetworkManager/nm-dhcp-client.action" pid=945 comm="apparmor_parser"
Oct 26 13:56:53 ctl01 systemd[1]: Starting OpenBSD Secure Shell server...
Oct 26 13:56:53 ctl01 kernel: [    5.119824] audit: type=1400 audit(1572098207.976:13): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/lib/NetworkManager/nm-dhcp-helper" pid=945 comm="apparmor_parser"
Oct 26 13:56:53 ctl01 kernel: [    5.119826] audit: type=1400 audit(1572098207.976:14): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/lib/connman/scripts/dhclient-script" pid=945 comm="apparmor_parser"
Oct 26 13:56:53 ctl01 kernel: [    8.248979] ISO 9660 Extensions: Microsoft Joliet Level 3
Oct 26 13:56:53 ctl01 kernel: [    8.255880] ISO 9660 Extensions: RRIP_1991A
Oct 26 13:56:53 ctl01 kernel: [    8.499875] br-mgmt: port 1(ens4) entered blocking state
Oct 26 13:56:53 ctl01 kernel: [    8.499879] br-mgmt: port 1(ens4) entered forwarding state
Oct 26 13:56:53 ctl01 kernel: [    8.500517] IPv6: ADDRCONF(NETDEV_UP): br-mgmt: link is not ready
Oct 26 13:56:53 ctl01 acpid: 1 rule loaded
Oct 26 13:56:53 ctl01 kernel: [    8.500534] IPv6: ADDRCONF(NETDEV_CHANGE): br-mgmt: link becomes ready
Oct 26 13:56:53 ctl01 kernel: [    8.588416] 8021q: 802.1Q VLAN Support v1.8
Oct 26 13:56:53 ctl01 kernel: [    8.588426] 8021q: adding VLAN 0 to HW filter on device ens3
Oct 26 13:56:53 ctl01 kernel: [    8.588498] 8021q: adding VLAN 0 to HW filter on device ens4
Oct 26 13:56:53 ctl01 kernel: [    8.588523] 8021q: adding VLAN 0 to HW filter on device ens5
Oct 26 13:56:53 ctl01 kernel: [   10.483369] new mount options do not match the existing superblock, will be ignored
Oct 26 13:56:53 ctl01 acpid: waiting for events: event logging is off
Oct 26 13:56:53 ctl01 systemd[1]: Started D-Bus System Message Bus.
Oct 26 13:56:53 ctl01 rsyslogd-2007: action 'action 11' suspended, next retry is Sat Oct 26 13:57:23 2019 [v8.16.0 try http://www.rsyslog.com/e/2007 ]
Oct 26 13:56:53 ctl01 smartd[2062]: smartd 6.5 2016-01-24 r4214 [x86_64-linux-4.15.0-66-generic] (local build)
Oct 26 13:56:53 ctl01 smartd[2062]: Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org
Oct 26 13:56:53 ctl01 smartd[2062]: Opened configuration file /etc/smartd.conf
Oct 26 13:56:53 ctl01 smartd[2062]: Drive: DEVICESCAN, implied '-a' Directive on line 21 of file /etc/smartd.conf
Oct 26 13:56:53 ctl01 smartd[2062]: Configuration file /etc/smartd.conf was parsed, found DEVICESCAN, scanning devices
Oct 26 13:56:53 ctl01 smartd[2062]: Problem creating device name scan list
Oct 26 13:56:53 ctl01 smartd[2062]: In the system's table of devices NO devices found to scan
Oct 26 13:56:53 ctl01 smartd[2062]: Unable to monitor any SMART enabled devices. Try debug (-d) option. Exiting...
Oct 26 13:56:53 ctl01 dbus[2084]: [system] AppArmor D-Bus mediation is enabled
Oct 26 13:56:53 ctl01 dbus[2084]: [system] Successfully activated service 'org.freedesktop.systemd1'
Oct 26 13:56:53 ctl01 systemd[1]: Starting Snappy daemon...
Oct 26 13:56:53 ctl01 systemd[1]: Starting Accounts Service...
Oct 26 13:56:53 ctl01 thermald[1967]: NO RAPL sysfs present
Oct 26 13:56:53 ctl01 thermald[1967]: 13 CPUID levels; family:model:stepping 0x6:3e:4 (6:62:4)
Oct 26 13:56:53 ctl01 thermald[1967]:  Need Linux PowerCap sysfs
Oct 26 13:56:53 ctl01 systemd[1]: Started System Logging Service.
Oct 26 13:56:53 ctl01 systemd[1]: Started /etc/rc.local Compatibility.
Oct 26 13:56:53 ctl01 systemd[1]: smartd.service: Main process exited, code=exited, status=17/n/a
Oct 26 13:56:53 ctl01 systemd[1]: smartd.service: Unit entered failed state.
Oct 26 13:56:53 ctl01 systemd[1]: smartd.service: Failed with result 'exit-code'.
Oct 26 13:56:53 ctl01 systemd[1]: Started OpenBSD Secure Shell server.
Oct 26 13:56:53 ctl01 sysfsutils[1972]:  * Setting sysfs variables...
Oct 26 13:56:53 ctl01 mdadm[2025]:  * Starting MD monitoring service mdadm --monitor
Oct 26 13:56:53 ctl01 systemd[1]: Started Thermal Daemon Service.
Oct 26 13:56:53 ctl01 systemd[1]: Started Login Service.
Oct 26 13:56:53 ctl01 systemd[1]: Started Unattended Upgrades Shutdown.
Oct 26 13:56:53 ctl01 iscsid: iSCSI daemon with pid=1955 started!
Oct 26 13:56:54 ctl01 sysfsutils[1972]:    ...done.
Oct 26 13:56:54 ctl01 systemd[1]: Started LSB: Set sysfs variables from /etc/sysfs.conf.
Oct 26 13:56:54 ctl01 thermald[1967]: Unsupported cpu model, use thermal-conf.xml file or run with --ignore-cpuid-check
Oct 26 13:56:54 ctl01 thermald[1967]: THD engine start failed
Oct 26 13:56:54 ctl01 iscsiadm[2155]: iscsiadm: No records found
Oct 26 13:56:54 ctl01 systemd[1]: Started Login to default iSCSI targets.
Oct 26 13:56:54 ctl01 mdadm[2025]:    ...done.
Oct 26 13:56:54 ctl01 systemd[1]: Started LSB: MD monitoring daemon.
Oct 26 13:56:54 ctl01 systemd[1]: Reached target Remote File Systems (Pre).
Oct 26 13:56:54 ctl01 systemd[1]: Reached target Remote File Systems.
Oct 26 13:56:54 ctl01 systemd[1]: Starting LSB: automatic crash report generation...
Oct 26 13:56:54 ctl01 systemd[1]: Starting Permit User Sessions...
Oct 26 13:56:54 ctl01 systemd[1]: Starting LSB: daemon to balance interrupts for SMP systems...
Oct 26 13:56:54 ctl01 systemd[1]: Started Permit User Sessions.
Oct 26 13:56:54 ctl01 irqbalance[2235]:  * Starting SMP IRQ Balancer: irqbalance
Oct 26 13:56:54 ctl01 dbus[2084]: [system] Activating via systemd: service name='org.freedesktop.PolicyKit1' unit='polkitd.service'
Oct 26 13:56:54 ctl01 systemd[1]: Started LSB: Record successful boot for GRUB.
Oct 26 13:56:54 ctl01 systemd[1]: Starting Authenticate and Authorize Users to Run Privileged Tasks...
Oct 26 13:56:54 ctl01 apport[2226]:  * Starting automatic crash report generation: apport
Oct 26 13:56:54 ctl01 systemd[1]: Starting Terminate Plymouth Boot Screen...
Oct 26 13:56:54 ctl01 irqbalance[2235]:    ...done.
Oct 26 13:56:54 ctl01 systemd[1]: Starting Hold until boot process finishes up...
Oct 26 13:56:54 ctl01 systemd[1]: Started LSB: daemon to balance interrupts for SMP systems.
Oct 26 13:56:54 ctl01 systemd[1]: Started Hold until boot process finishes up.
Oct 26 13:56:54 ctl01 systemd[1]: Started Terminate Plymouth Boot Screen.
Oct 26 13:56:54 ctl01 apport[2226]:    ...done.
Oct 26 13:56:54 ctl01 systemd[1]: Started LSB: automatic crash report generation.
Oct 26 13:56:54 ctl01 systemd[1]: Starting Set console scheme...
Oct 26 13:56:54 ctl01 systemd[1]: Started Getty on tty1.
Oct 26 13:56:54 ctl01 systemd[1]: Started Serial Getty on ttyS0.
Oct 26 13:56:54 ctl01 systemd[1]: Reached target Login Prompts.
Oct 26 13:56:54 ctl01 systemd[1]: Started Set console scheme.
Oct 26 13:56:55 ctl01 polkitd[2293]: started daemon version 0.105 using authority implementation `local' version `0.105'
Oct 26 13:56:55 ctl01 dbus[2084]: [system] Successfully activated service 'org.freedesktop.PolicyKit1'
Oct 26 13:56:55 ctl01 systemd[1]: Started Authenticate and Authorize Users to Run Privileged Tasks.
Oct 26 13:56:55 ctl01 accounts-daemon[2136]: started daemon version 0.6.40
Oct 26 13:56:55 ctl01 systemd[1]: Started Accounts Service.
Oct 26 13:56:55 ctl01 systemd[1]: Started LXD - container startup/shutdown.
Oct 26 13:56:55 ctl01 ovs-ctl[2006]:  * Starting ovsdb-server
Oct 26 13:56:55 ctl01 ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl --no-wait -- init -- set Open_vSwitch . db-version=7.12.1
Oct 26 13:56:55 ctl01 snapd[2126]: AppArmor status: apparmor is enabled and all features are available
Oct 26 13:56:55 ctl01 snapd[2126]: patch.go:64: Patching system state level 6 to sublevel 1...
Oct 26 13:56:55 ctl01 ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl --no-wait set Open_vSwitch . ovs-version=2.5.5 "external-ids:system-id=\"49a27502-2a2a-45e5-a45d-6bfe2da26bab\"" "system-type=\"Ubuntu\"" "system-version=\"16.04-xenial\""
Oct 26 13:56:55 ctl01 ovs-ctl[2006]:  * Configuring Open vSwitch system IDs
Oct 26 13:56:55 ctl01 snapd[2126]: daemon.go:338: started snapd/2.40 (series 16; classic) ubuntu/16.04 (amd64) linux/4.15.0-66-generic.
Oct 26 13:56:55 ctl01 ovs-ctl[2006]:  * Inserting openvswitch module
Oct 26 13:56:55 ctl01 kernel: [   12.992079] openvswitch: Open vSwitch switching datapath
Oct 26 13:56:55 ctl01 ovs-ctl[2006]:  * Starting ovs-vswitchd
Oct 26 13:56:55 ctl01 ovs-ctl[2006]:  * Enabling remote OVSDB managers
Oct 26 13:56:55 ctl01 systemd[1]: Started Open vSwitch Internal Unit.
Oct 26 13:56:55 ctl01 systemd[1]: Starting Open vSwitch...
Oct 26 13:56:55 ctl01 systemd[1]: Started Open vSwitch.
Oct 26 13:56:55 ctl01 systemd[1]: Started Snappy daemon.
Oct 26 13:56:55 ctl01 systemd[1]: Starting Wait until snapd is fully seeded...
Oct 26 13:56:55 ctl01 systemd[1]: Started Wait until snapd is fully seeded.
Oct 26 13:56:55 ctl01 systemd[1]: Starting Apply the settings specified in cloud-config...
Oct 26 13:56:56 ctl01 cloud-init[2497]: Cloud-init v. 19.2-36-g059d049c-0ubuntu2~16.04.1 running 'modules:config' at Sat, 26 Oct 2019 13:56:56 +0000. Up 13.39 seconds.
Oct 26 13:56:56 ctl01 systemd[1]: Started The Salt Minion.
Oct 26 13:56:56 ctl01 systemd[1]: Reached target Multi-User System.
Oct 26 13:56:56 ctl01 systemd[1]: Reached target Graphical Interface.
Oct 26 13:56:56 ctl01 systemd[1]: Starting Update UTMP about System Runlevel Changes...
Oct 26 13:56:56 ctl01 systemd[1]: Started Update UTMP about System Runlevel Changes.
Oct 26 13:56:56 ctl01 systemd[1]: Started Apply the settings specified in cloud-config.
Oct 26 13:56:56 ctl01 systemd[1]: Starting Execute cloud user/final scripts...
Oct 26 13:56:57 ctl01 cloud-init[2543]: Cloud-init v. 19.2-36-g059d049c-0ubuntu2~16.04.1 running 'modules:final' at Sat, 26 Oct 2019 13:56:56 +0000. Up 13.91 seconds.
Oct 26 13:56:57 ctl01 cloud-init[2543]: Cloud-init v. 19.2-36-g059d049c-0ubuntu2~16.04.1 finished at Sat, 26 Oct 2019 13:56:57 +0000. Datasource DataSourceNoCloud [seed=/dev/sr0][dsmode=net].  Up 14.20 seconds
Oct 26 13:56:57 ctl01 systemd[1]: Started Execute cloud user/final scripts.
Oct 26 13:56:57 ctl01 systemd[1]: Reached target Cloud-init target.
Oct 26 13:56:57 ctl01 systemd[1]: Startup finished in 2.624s (kernel) + 11.624s (userspace) = 14.249s.
Oct 26 13:57:00 ctl01 snapd[2126]: daemon.go:576: gracefully waiting for running hooks
Oct 26 13:57:00 ctl01 snapd[2126]: daemon.go:578: done waiting for running hooks
Oct 26 13:57:00 ctl01 snapd[2126]: daemon stop requested to wait for socket activation
Oct 26 13:57:03 ctl01 kernel: [   20.471533] random: crng init done
Oct 26 13:57:03 ctl01 kernel: [   20.471539] random: 7 urandom warning(s) missed due to ratelimiting
Oct 26 13:57:16 ctl01 systemd[1]: Time has been changed
Oct 26 13:57:16 ctl01 systemd-timesyncd[573]: Synchronized to time server 91.189.94.4:123 (ntp.ubuntu.com).
Oct 26 13:57:25 ctl01 systemd[1]: Started /usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold -o DPkg::Options::=--force-confdef dist-upgrade.
Oct 26 13:57:45 ctl01 systemd[1]: Started /usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold -o DPkg::Options::=--force-confdef install python-oauth python-m2crypto.
Oct 26 13:57:49 ctl01 systemd[1]: Reloading.
Oct 26 13:57:49 ctl01 systemd[1]: Started ACPI event daemon.
Oct 26 13:57:50 ctl01 salt-minion[1962]: [WARNING ] The function "module.run" is using its deprecated version and will expire in version "Sodium".
Oct 26 13:57:50 ctl01 salt-minion[1962]: ......++
Oct 26 13:57:50 ctl01 salt-minion[1962]: .................................++
Oct 26 13:57:52 ctl01 salt-minion[1962]: [WARNING ] State for file: /etc/kubernetes/ssl/ca-kubernetes.crt - Neither 'source' nor 'contents' nor 'contents_pillar' nor 'contents_grains' was defined, yet 'replace' was set to 'True'. As there is no source to replace the file with, 'replace' has been set to 'False' to avoid reading the file unnecessarily.
Oct 26 13:57:52 ctl01 salt-minion[1962]: ...........................................++
Oct 26 13:57:53 ctl01 salt-minion[1962]: ..........................................................++
Oct 26 13:57:54 ctl01 salt-minion[1962]: .........................................++
Oct 26 13:57:54 ctl01 salt-minion[1962]: ...................................++
Oct 26 13:57:55 ctl01 salt-minion[1962]: [WARNING ] State for file: /var/lib/etcd/ca.pem - Neither 'source' nor 'contents' nor 'contents_pillar' nor 'contents_grains' was defined, yet 'replace' was set to 'True'. As there is no source to replace the file with, 'replace' has been set to 'False' to avoid reading the file unnecessarily.
Oct 26 13:57:55 ctl01 salt-minion[1962]: ............................................++
Oct 26 13:57:56 ctl01 salt-minion[1962]: .........................................................................++
Oct 26 13:57:56 ctl01 salt-minion[1962]: ...............++
Oct 26 13:57:57 ctl01 salt-minion[1962]: .......................................................................................................................................................++
Oct 26 13:57:59 ctl01 salt-minion[1962]: .......................................................................................................++
Oct 26 13:57:59 ctl01 salt-minion[1962]: ...++
Oct 26 13:58:00 ctl01 salt-minion[1962]: ........................................++
Oct 26 13:58:00 ctl01 salt-minion[1962]: ...............................++
Oct 26 13:58:01 ctl01 salt-minion[1962]: ..................................++
Oct 26 13:58:02 ctl01 salt-minion[1962]: ..............................................................................................................................................................++
Oct 26 13:58:02 ctl01 systemd[1]: Started /usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold -o DPkg::Options::=--force-confdef install ntp.
Oct 26 13:58:05 ctl01 systemd[1]: Reloading.
Oct 26 13:58:05 ctl01 systemd[1]: Started ACPI event daemon.
Oct 26 13:58:06 ctl01 systemd[1]: Reloading.
Oct 26 13:58:06 ctl01 systemd[1]: Started ACPI event daemon.
Oct 26 13:58:06 ctl01 systemd[1]: Reloading.
Oct 26 13:58:06 ctl01 systemd[1]: Started ACPI event daemon.
Oct 26 13:58:06 ctl01 systemd[1]: Starting LSB: Start NTP daemon...
Oct 26 13:58:06 ctl01 ntp[4214]:  * Starting NTP server ntpd
Oct 26 13:58:06 ctl01 ntpd[4234]: ntpd 4.2.8p4@1.3265-o Fri Jul  6 20:10:51 UTC 2018 (1): Starting
Oct 26 13:58:06 ctl01 ntpd[4234]: Command line: /usr/sbin/ntpd -p /var/run/ntpd.pid -g -u 112:116
Oct 26 13:58:06 ctl01 ntp[4214]:    ...done.
Oct 26 13:58:06 ctl01 systemd[1]: Started LSB: Start NTP daemon.
Oct 26 13:58:07 ctl01 kernel: [   84.695416] audit: type=1400 audit(1572098287.003:15): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/sbin/ntpd" pid=4247 comm="apparmor_parser"
Oct 26 13:58:07 ctl01 ntpd[4238]: proto: precision = 0.061 usec (-24)
Oct 26 13:58:07 ctl01 ntpd[4238]: Listen and drop on 0 v6wildcard [::]:123
Oct 26 13:58:07 ctl01 ntpd[4238]: Listen and drop on 1 v4wildcard 0.0.0.0:123
Oct 26 13:58:07 ctl01 ntpd[4238]: Listen normally on 2 lo 127.0.0.1:123
Oct 26 13:58:07 ctl01 ntpd[4238]: Listen normally on 3 ens3 192.168.11.21:123
Oct 26 13:58:07 ctl01 ntpd[4238]: Listen normally on 4 br-mgmt 172.16.10.36:123
Oct 26 13:58:07 ctl01 ntpd[4238]: Listen normally on 5 lo [::1]:123
Oct 26 13:58:07 ctl01 ntpd[4238]: Listen normally on 6 ens3 [fe80::5054:ff:fed1:44f2%2]:123
Oct 26 13:58:07 ctl01 ntpd[4238]: Listen normally on 7 ens4 [fe80::5054:ff:fecc:bfd7%3]:123
Oct 26 13:58:07 ctl01 ntpd[4238]: Listen normally on 8 ens5 [fe80::5054:ff:fedd:4a72%4]:123
Oct 26 13:58:07 ctl01 ntpd[4238]: Listen normally on 9 br-mgmt [fe80::5054:ff:fecc:bfd7%6]:123
Oct 26 13:58:07 ctl01 ntpd[4238]: Listen normally on 10 ens5.1000 [fe80::5054:ff:fedd:4a72%7]:123
Oct 26 13:58:07 ctl01 ntpd[4238]: Listening on routing socket on fd #27 for interface updates
Oct 26 13:58:07 ctl01 systemd[1]: Reloading.
Oct 26 13:58:07 ctl01 systemd[1]: Started ACPI event daemon.
Oct 26 13:58:09 ctl01 systemd[1]: Started /bin/systemctl restart ntp.service.
Oct 26 13:58:09 ctl01 systemd[1]: Stopping LSB: Start NTP daemon...
Oct 26 13:58:09 ctl01 ntp[4371]:  * Stopping NTP server ntpd
Oct 26 13:58:09 ctl01 ntp[4371]:    ...done.
Oct 26 13:58:09 ctl01 ntpd[4238]: ntpd exiting on signal 15 (Terminated)
Oct 26 13:58:09 ctl01 systemd[1]: Stopped LSB: Start NTP daemon.
Oct 26 13:58:09 ctl01 systemd[1]: Starting LSB: Start NTP daemon...
Oct 26 13:58:09 ctl01 ntp[4382]:  * Starting NTP server ntpd
Oct 26 13:58:09 ctl01 ntpd[4405]: ntpd 4.2.8p4@1.3265-o Fri Jul  6 20:10:51 UTC 2018 (1): Starting
Oct 26 13:58:09 ctl01 ntpd[4405]: Command line: /usr/sbin/ntpd -p /var/run/ntpd.pid -g -u 112:116
Oct 26 13:58:09 ctl01 ntp[4382]:    ...done.
Oct 26 13:58:09 ctl01 systemd[1]: Started LSB: Start NTP daemon.
Oct 26 13:58:09 ctl01 ntpd[4408]: proto: precision = 0.062 usec (-24)
Oct 26 13:58:09 ctl01 ntpd[4408]: restrict 0.0.0.0: KOD does nothing without LIMITED.
Oct 26 13:58:09 ctl01 ntpd[4408]: restrict ::: KOD does nothing without LIMITED.
Oct 26 13:58:09 ctl01 ntpd[4408]: switching logging to file /var/log/ntp.log
Oct 26 13:58:12 ctl01 salt-minion[1962]: [INFO    ] Executing command ['systemctl', 'status', 'salt-minion.service', '-n', '0'] in directory '/root'
Oct 26 13:58:12 ctl01 salt-minion[1962]: [INFO    ] Executing command ['systemd-run', '--scope', 'systemctl', 'restart', 'salt-minion.service'] in directory '/root'
Oct 26 13:58:12 ctl01 systemd[1]: Started /bin/systemctl restart salt-minion.service.
Oct 26 13:58:12 ctl01 systemd[1]: Stopping The Salt Minion...
Oct 26 13:58:12 ctl01 salt-minion[1962]: [WARNING ] Minion received a SIGTERM. Exiting.
Oct 26 13:58:13 ctl01 salt-minion[1962]: The Salt Minion is shutdown. Minion received a SIGTERM. Exited.
Oct 26 13:58:13 ctl01 systemd[1]: Stopped The Salt Minion.
Oct 26 13:58:13 ctl01 systemd[1]: Starting The Salt Minion...
Oct 26 13:58:13 ctl01 systemd[1]: Started The Salt Minion.
Oct 26 13:58:13 ctl01 salt-minion[1962]: local:
Oct 26 13:58:13 ctl01 salt-minion[1962]:     True
Oct 26 13:58:13 ctl01 salt-minion[4526]: [INFO    ] Setting up the Salt Minion "ctl01.mcp-k8s-calico-noha.local"
Oct 26 13:58:14 ctl01 salt-minion[4526]: [INFO    ] Starting up the Salt Minion
Oct 26 13:58:14 ctl01 salt-minion[4526]: [INFO    ] Starting pull socket on /var/run/salt/minion/minion_event_1899fe9592_pull.ipc
Oct 26 13:58:14 ctl01 salt-minion[4526]: [INFO    ] Creating minion process manager
Oct 26 13:58:16 ctl01 salt-minion[4526]: [INFO    ] Executing command ['date', '+%z'] in directory '/root'
Oct 26 13:58:16 ctl01 salt-minion[4526]: [INFO    ] Updating job settings for scheduled job: __mine_interval
Oct 26 13:58:16 ctl01 salt-minion[4526]: [INFO    ] Added mine.update to scheduler
Oct 26 13:58:16 ctl01 salt-minion[4526]: [INFO    ] Minion is starting as user 'root'
Oct 26 13:58:16 ctl01 salt-minion[4526]: [INFO    ] Minion is ready to receive requests!
Oct 26 13:58:25 ctl01 salt-minion[4526]: [INFO    ] User sudo_ubuntu Executing command state.sls with jid 20191026135825793262
Oct 26 13:58:25 ctl01 salt-minion[4526]: [INFO    ] Starting a new job with PID 4617
Oct 26 13:58:26 ctl01 salt-minion[4526]: [INFO    ] Loading fresh modules for state activity
Oct 26 13:58:26 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'etcd/server/service.sls'
Oct 26 13:58:27 ctl01 salt-minion[4526]: [INFO    ] Executing command '. /var/lib/etcd/configenv; etcdctl cluster-health > /dev/null 2>&1; echo $?' in directory '/root'
Oct 26 13:58:27 ctl01 salt-minion[4526]: [ERROR   ] Command '. /var/lib/etcd/configenv; etcdctl cluster-health > /dev/null 2>&1; echo $?' failed with return code: 2
Oct 26 13:58:27 ctl01 salt-minion[4526]: [ERROR   ] stdout: /bin/sh: 1: .: Can't open /var/lib/etcd/configenv
Oct 26 13:58:27 ctl01 salt-minion[4526]: [ERROR   ] retcode: 2
Oct 26 13:58:27 ctl01 salt-minion[4526]: [INFO    ] Running state [etcd_support_packages] at time 13:58:27.742575
Oct 26 13:58:27 ctl01 salt-minion[4526]: [INFO    ] Executing state pkg.installed for [etcd_support_packages]
Oct 26 13:58:27 ctl01 salt-minion[4526]: [INFO    ] Executing command ['dpkg-query', '--showformat', '${Status} ${Package} ${Version} ${Architecture}', '-W'] in directory '/root'
Oct 26 13:58:28 ctl01 salt-minion[4526]: [INFO    ] Executing command ['apt-cache', '-q', 'policy', 'python-etcd'] in directory '/root'
Oct 26 13:58:28 ctl01 salt-minion[4526]: [INFO    ] Executing command ['apt-get', '-q', 'update'] in directory '/root'
Oct 26 13:58:30 ctl01 salt-minion[4526]: [INFO    ] Executing command ['dpkg', '--get-selections', '*'] in directory '/root'
Oct 26 13:58:30 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemd-run', '--scope', 'apt-get', '-q', '-y', '-o', 'DPkg::Options::=--force-confold', '-o', 'DPkg::Options::=--force-confdef', 'install', 'python-etcd'] in directory '/root'
Oct 26 13:58:30 ctl01 systemd[1]: Started /usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold -o DPkg::Options::=--force-confdef install python-etcd.
Oct 26 13:58:32 ctl01 salt-minion[4526]: [INFO    ] Executing command ['dpkg-query', '--showformat', '${Status} ${Package} ${Version} ${Architecture}', '-W'] in directory '/root'
Oct 26 13:58:32 ctl01 salt-minion[4526]: [INFO    ] Made the following changes:
Oct 26 13:58:32 ctl01 salt-minion[4526]: 'python-etcd' changed from 'absent' to '0.4.2-2'
Oct 26 13:58:32 ctl01 salt-minion[4526]: [INFO    ] Loading fresh modules for state activity
Oct 26 13:58:33 ctl01 salt-minion[4526]: [INFO    ] Completed state [etcd_support_packages] at time 13:58:33.063681 duration_in_ms=5321.106
Oct 26 13:58:33 ctl01 salt-minion[4526]: [INFO    ] Running state [/tmp/etcd/bin] at time 13:58:33.068757
Oct 26 13:58:33 ctl01 salt-minion[4526]: [INFO    ] Executing state archive.extracted for [/tmp/etcd/bin]
Oct 26 13:58:35 ctl01 salt-minion[4526]: [INFO    ] Executing command ['tar', 'x', '--strip=1', '-f', '/var/cache/salt/minion/extrn_files/base/github.com/etcd-io/etcd/releases/download/v3.3.12/etcd-v3.3.12-linux-amd64.tar.gz'] in directory '/tmp/etcd/bin/'
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] Executing command ['tar', '--version'] in directory '/root'
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] {'extracted_files': 'no tar output so far', 'directories_created': ['/tmp/etcd/bin/']}
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] Completed state [/tmp/etcd/bin] at time 13:58:36.178604 duration_in_ms=3109.842
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] Running state [/usr/local/bin/etcd] at time 13:58:36.179614
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/usr/local/bin/etcd]
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:58:36 ctl01 salt-minion[4526]: New file
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] Completed state [/usr/local/bin/etcd] at time 13:58:36.516434 duration_in_ms=336.82
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] Running state [/usr/local/bin/etcdctl] at time 13:58:36.516805
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/usr/local/bin/etcdctl]
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:58:36 ctl01 salt-minion[4526]: New file
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] Completed state [/usr/local/bin/etcdctl] at time 13:58:36.679338 duration_in_ms=162.532
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] Running state [etcd] at time 13:58:36.681479
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] Executing state user.present for [etcd]
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] User etcd is present and up to date
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] Completed state [etcd] at time 13:58:36.692581 duration_in_ms=11.101
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/systemd/system/etcd.service] at time 13:58:36.692796
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/systemd/system/etcd.service]
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'etcd/files/systemd/etcd.service'
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:58:36 ctl01 salt-minion[4526]: New file
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/systemd/system/etcd.service] at time 13:58:36.725534 duration_in_ms=32.737
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/default/etcd] at time 13:58:36.725782
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/default/etcd]
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'etcd/files/default'
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:58:36 ctl01 salt-minion[4526]: New file
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/default/etcd] at time 13:58:36.813838 duration_in_ms=88.055
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] Running state [/var/lib/etcd/] at time 13:58:36.814063
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] Executing state file.directory for [/var/lib/etcd/]
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] Directory /var/lib/etcd is in the correct state
Oct 26 13:58:36 ctl01 salt-minion[4526]: Directory /var/lib/etcd updated
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] Completed state [/var/lib/etcd/] at time 13:58:36.818833 duration_in_ms=4.77
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] Running state [/var/lib/etcd/configenv] at time 13:58:36.819365
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/var/lib/etcd/configenv]
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'etcd/files/configenv'
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:58:36 ctl01 salt-minion[4526]: New file
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] Completed state [/var/lib/etcd/configenv] at time 13:58:36.870852 duration_in_ms=51.486
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] Running state [/var/lib/etcd/configenvv3] at time 13:58:36.871209
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/var/lib/etcd/configenvv3]
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'etcd/files/configenvv3'
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:58:36 ctl01 salt-minion[4526]: New file
Oct 26 13:58:36 ctl01 salt-minion[4526]: [INFO    ] Completed state [/var/lib/etcd/configenvv3] at time 13:58:36.922805 duration_in_ms=51.596
Oct 26 13:58:37 ctl01 salt-minion[4526]: [INFO    ] Running state [etcd] at time 13:58:37.403510
Oct 26 13:58:37 ctl01 salt-minion[4526]: [INFO    ] Executing state service.running for [etcd]
Oct 26 13:58:37 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'status', 'etcd.service', '-n', '0'] in directory '/root'
Oct 26 13:58:37 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-active', 'etcd.service'] in directory '/root'
Oct 26 13:58:37 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-enabled', 'etcd.service'] in directory '/root'
Oct 26 13:58:37 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemd-run', '--scope', 'systemctl', 'start', 'etcd.service'] in directory '/root'
Oct 26 13:58:37 ctl01 systemd[1]: Started /bin/systemctl start etcd.service.
Oct 26 13:58:37 ctl01 systemd[1]: Starting etcd - highly-available key value store...
Oct 26 13:58:37 ctl01 etcd[5203]: recognized and used environment variable ETCD_ADVERTISE_CLIENT_URLS=https://172.16.10.36:4001
Oct 26 13:58:37 ctl01 etcd[5203]: recognized and used environment variable ETCD_CERT_FILE=/var/lib/etcd/etcd-server.crt
Oct 26 13:58:37 ctl01 etcd[5203]: recognized and used environment variable ETCD_CLIENT_CERT_AUTH=true
Oct 26 13:58:37 ctl01 etcd[5203]: recognized and used environment variable ETCD_DATA_DIR=/var/lib/etcd/default
Oct 26 13:58:37 ctl01 etcd[5203]: recognized and used environment variable ETCD_ELECTION_TIMEOUT=5000
Oct 26 13:58:37 ctl01 etcd[5203]: recognized and used environment variable ETCD_HEARTBEAT_INTERVAL=250
Oct 26 13:58:37 ctl01 etcd[5203]: recognized and used environment variable ETCD_INITIAL_ADVERTISE_PEER_URLS=https://172.16.10.36:2380
Oct 26 13:58:37 ctl01 etcd[5203]: recognized and used environment variable ETCD_INITIAL_CLUSTER=ctl01=https://172.16.10.36:2380
Oct 26 13:58:37 ctl01 etcd[5203]: recognized and used environment variable ETCD_INITIAL_CLUSTER_STATE=new
Oct 26 13:58:37 ctl01 etcd[5203]: recognized and used environment variable ETCD_INITIAL_CLUSTER_TOKEN=IN7KaRMSo3xkGxkjAAPtkRkAgqN4ZNRq
Oct 26 13:58:37 ctl01 etcd[5203]: recognized and used environment variable ETCD_KEY_FILE=/var/lib/etcd/etcd-server.key
Oct 26 13:58:37 ctl01 etcd[5203]: recognized and used environment variable ETCD_LISTEN_CLIENT_URLS=https://172.16.10.36:4001,http://127.0.0.1:4001
Oct 26 13:58:37 ctl01 etcd[5203]: recognized and used environment variable ETCD_LISTEN_PEER_URLS=https://172.16.10.36:2380
Oct 26 13:58:37 ctl01 etcd[5203]: recognized and used environment variable ETCD_NAME=ctl01
Oct 26 13:58:37 ctl01 etcd[5203]: recognized and used environment variable ETCD_PEER_CERT_FILE=/var/lib/etcd/etcd-server.crt
Oct 26 13:58:37 ctl01 etcd[5203]: recognized and used environment variable ETCD_PEER_CLIENT_CERT_AUTH=true
Oct 26 13:58:37 ctl01 etcd[5203]: recognized and used environment variable ETCD_PEER_KEY_FILE=/var/lib/etcd/etcd-server.key
Oct 26 13:58:37 ctl01 etcd[5203]: recognized and used environment variable ETCD_PEER_TRUSTED_CA_FILE=/var/lib/etcd/ca.pem
Oct 26 13:58:37 ctl01 etcd[5203]: recognized and used environment variable ETCD_TRUSTED_CA_FILE=/var/lib/etcd/ca.pem
Oct 26 13:58:37 ctl01 etcd[5203]: setting maximum number of CPUs to 8, total number of available CPUs is 8
Oct 26 13:58:37 ctl01 etcd[5203]: peerTLS: cert = /var/lib/etcd/etcd-server.crt, key = /var/lib/etcd/etcd-server.key, ca = , trusted-ca = /var/lib/etcd/ca.pem, client-cert-auth = true, crl-file = 
Oct 26 13:58:37 ctl01 etcd[5203]: listening for peers on https://172.16.10.36:2380
Oct 26 13:58:37 ctl01 etcd[5203]: The scheme of client url http://127.0.0.1:4001 is HTTP while peer key/cert files are presented. Ignored key/cert files.
Oct 26 13:58:37 ctl01 etcd[5203]: The scheme of client url http://127.0.0.1:4001 is HTTP while client cert auth (--client-cert-auth) is enabled. Ignored client cert auth for this url.
Oct 26 13:58:37 ctl01 etcd[5203]: listening for client requests on 127.0.0.1:4001
Oct 26 13:58:37 ctl01 etcd[5203]: listening for client requests on 172.16.10.36:4001
Oct 26 13:58:37 ctl01 etcd[5203]: name = ctl01
Oct 26 13:58:37 ctl01 etcd[5203]: data dir = /var/lib/etcd/default
Oct 26 13:58:37 ctl01 etcd[5203]: member dir = /var/lib/etcd/default/member
Oct 26 13:58:37 ctl01 etcd[5203]: heartbeat = 250ms
Oct 26 13:58:37 ctl01 etcd[5203]: election = 5000ms
Oct 26 13:58:37 ctl01 etcd[5203]: snapshot count = 100000
Oct 26 13:58:37 ctl01 etcd[5203]: advertise client URLs = https://172.16.10.36:4001
Oct 26 13:58:37 ctl01 etcd[5203]: initial advertise peer URLs = https://172.16.10.36:2380
Oct 26 13:58:37 ctl01 etcd[5203]: initial cluster = ctl01=https://172.16.10.36:2380
Oct 26 13:58:37 ctl01 etcd[5203]: starting member 411212f5fea59a9f in cluster 8f5ffbb80be06f34
Oct 26 13:58:37 ctl01 etcd[5203]: 411212f5fea59a9f became follower at term 0
Oct 26 13:58:37 ctl01 etcd[5203]: newRaft 411212f5fea59a9f [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]
Oct 26 13:58:37 ctl01 etcd[5203]: 411212f5fea59a9f became follower at term 1
Oct 26 13:58:37 ctl01 etcd[5203]: simple token is not cryptographically signed
Oct 26 13:58:37 ctl01 etcd[5203]: starting server... [version: 3.3.12, cluster version: to_be_decided]
Oct 26 13:58:37 ctl01 etcd[5203]: 411212f5fea59a9f as single-node; fast-forwarding 19 ticks (election ticks 20)
Oct 26 13:58:37 ctl01 etcd[5203]: added member 411212f5fea59a9f [https://172.16.10.36:2380] to cluster 8f5ffbb80be06f34
Oct 26 13:58:37 ctl01 etcd[5203]: ClientTLS: cert = /var/lib/etcd/etcd-server.crt, key = /var/lib/etcd/etcd-server.key, ca = , trusted-ca = /var/lib/etcd/ca.pem, client-cert-auth = true, crl-file = 
Oct 26 13:58:40 ctl01 salt-minion[4526]: [INFO    ] User sudo_ubuntu Executing command saltutil.find_job with jid 20191026135840864291
Oct 26 13:58:40 ctl01 salt-minion[4526]: [INFO    ] Starting a new job with PID 5233
Oct 26 13:58:40 ctl01 salt-minion[4526]: [INFO    ] Returning information for job: 20191026135840864291
Oct 26 13:58:41 ctl01 etcd[5203]: 411212f5fea59a9f is starting a new election at term 1
Oct 26 13:58:41 ctl01 etcd[5203]: 411212f5fea59a9f became candidate at term 2
Oct 26 13:58:41 ctl01 etcd[5203]: 411212f5fea59a9f received MsgVoteResp from 411212f5fea59a9f at term 2
Oct 26 13:58:41 ctl01 etcd[5203]: 411212f5fea59a9f became leader at term 2
Oct 26 13:58:41 ctl01 etcd[5203]: raft.node: 411212f5fea59a9f elected leader 411212f5fea59a9f at term 2
Oct 26 13:58:41 ctl01 etcd[5203]: published {Name:ctl01 ClientURLs:[https://172.16.10.36:4001]} to cluster 8f5ffbb80be06f34
Oct 26 13:58:41 ctl01 etcd[5203]: ready to serve client requests
Oct 26 13:58:41 ctl01 etcd[5203]: ready to serve client requests
Oct 26 13:58:41 ctl01 etcd[5203]: setting up the initial cluster version to 3.3
Oct 26 13:58:41 ctl01 systemd[1]: Started etcd - highly-available key value store.
Oct 26 13:58:41 ctl01 etcd[5203]: serving insecure client requests on 127.0.0.1:4001, this is strongly discouraged!
Oct 26 13:58:41 ctl01 etcd[5203]: set the initial cluster version to 3.3
Oct 26 13:58:41 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-active', 'etcd.service'] in directory '/root'
Oct 26 13:58:41 ctl01 etcd[5203]: enabled capabilities for version 3.3
Oct 26 13:58:41 ctl01 etcd[5203]: serving client requests on 172.16.10.36:4001
Oct 26 13:58:41 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-enabled', 'etcd.service'] in directory '/root'
Oct 26 13:58:41 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-enabled', 'etcd.service'] in directory '/root'
Oct 26 13:58:41 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemd-run', '--scope', 'systemctl', 'enable', 'etcd.service'] in directory '/root'
Oct 26 13:58:41 ctl01 systemd[1]: Started /bin/systemctl enable etcd.service.
Oct 26 13:58:41 ctl01 systemd[1]: Reloading.
Oct 26 13:58:41 ctl01 systemd[1]: Started ACPI event daemon.
Oct 26 13:58:41 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-enabled', 'etcd.service'] in directory '/root'
Oct 26 13:58:41 ctl01 salt-minion[4526]: [INFO    ] {'etcd': True}
Oct 26 13:58:41 ctl01 salt-minion[4526]: [INFO    ] Completed state [etcd] at time 13:58:41.265768 duration_in_ms=3862.257
Oct 26 13:58:41 ctl01 salt-minion[4526]: [INFO    ] Returning information for job: 20191026135825793262
Oct 26 13:58:41 ctl01 salt-minion[4526]: [INFO    ] User sudo_ubuntu Executing command cmd.run with jid 20191026135841937351
Oct 26 13:58:41 ctl01 salt-minion[4526]: [INFO    ] Starting a new job with PID 5285
Oct 26 13:58:41 ctl01 salt-minion[4526]: [INFO    ] Executing command '. /var/lib/etcd/configenv && etcdctl cluster-health' in directory '/root'
Oct 26 13:58:42 ctl01 salt-minion[4526]: [INFO    ] Returning information for job: 20191026135841937351
Oct 26 13:58:42 ctl01 salt-minion[4526]: [INFO    ] User sudo_ubuntu Executing command state.sls with jid 20191026135842743315
Oct 26 13:58:42 ctl01 salt-minion[4526]: [INFO    ] Starting a new job with PID 5299
Oct 26 13:58:49 ctl01 salt-minion[4526]: [INFO    ] Loading fresh modules for state activity
Oct 26 13:58:49 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/master/kube-addons.sls'
Oct 26 13:58:49 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:58:49 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:58:49 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:58:49 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:58:49 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:58:49 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:58:49 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/addons] at time 13:58:49.641855
Oct 26 13:58:49 ctl01 salt-minion[4526]: [INFO    ] Executing state file.directory for [/etc/kubernetes/addons]
Oct 26 13:58:49 ctl01 salt-minion[4526]: [INFO    ] {'/etc/kubernetes/addons': 'New Dir'}
Oct 26 13:58:49 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/addons] at time 13:58:49.648527 duration_in_ms=6.673
Oct 26 13:58:49 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/addons/calico/calico-kube-controllers.yml] at time 13:58:49.648770
Oct 26 13:58:49 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/kubernetes/addons/calico/calico-kube-controllers.yml]
Oct 26 13:58:49 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/kube-addons/calico/calico-kube-controllers.yml'
Oct 26 13:58:49 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:58:49 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:58:49 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:58:49 ctl01 salt-minion[4526]: New file
Oct 26 13:58:49 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/addons/calico/calico-kube-controllers.yml] at time 13:58:49.783299 duration_in_ms=134.529
Oct 26 13:58:49 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/addons/calico/calico-rbac.yml] at time 13:58:49.783579
Oct 26 13:58:49 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/kubernetes/addons/calico/calico-rbac.yml]
Oct 26 13:58:49 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/kube-addons/calico/calico-rbac.yml'
Oct 26 13:58:49 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:58:49 ctl01 salt-minion[4526]: New file
Oct 26 13:58:49 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/addons/calico/calico-rbac.yml] at time 13:58:49.839290 duration_in_ms=55.71
Oct 26 13:58:49 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/addons/netchecker/netchecker-svc.yml] at time 13:58:49.839719
Oct 26 13:58:49 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/kubernetes/addons/netchecker/netchecker-svc.yml]
Oct 26 13:58:49 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/kube-addons/netchecker/netchecker-svc.yml'
Oct 26 13:58:49 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:58:49 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:58:49 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:58:49 ctl01 salt-minion[4526]: New file
Oct 26 13:58:49 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/addons/netchecker/netchecker-svc.yml] at time 13:58:49.960863 duration_in_ms=121.142
Oct 26 13:58:49 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/addons/netchecker/netchecker-server.yml] at time 13:58:49.961604
Oct 26 13:58:49 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/kubernetes/addons/netchecker/netchecker-server.yml]
Oct 26 13:58:49 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/kube-addons/netchecker/netchecker-server.yml'
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:58:50 ctl01 salt-minion[4526]: New file
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/addons/netchecker/netchecker-server.yml] at time 13:58:50.121945 duration_in_ms=160.34
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/addons/netchecker/netchecker-agent.yml] at time 13:58:50.122465
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/kubernetes/addons/netchecker/netchecker-agent.yml]
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/kube-addons/netchecker/netchecker-agent.yml'
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:58:50 ctl01 salt-minion[4526]: New file
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/addons/netchecker/netchecker-agent.yml] at time 13:58:50.241246 duration_in_ms=118.78
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/addons/netchecker/netchecker-serviceaccount.yml] at time 13:58:50.241748
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/kubernetes/addons/netchecker/netchecker-serviceaccount.yml]
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/kube-addons/netchecker/netchecker-serviceaccount.yml'
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:58:50 ctl01 salt-minion[4526]: New file
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/addons/netchecker/netchecker-serviceaccount.yml] at time 13:58:50.276218 duration_in_ms=34.47
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/addons/netchecker/netchecker-roles.yml] at time 13:58:50.276449
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/kubernetes/addons/netchecker/netchecker-roles.yml]
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/kube-addons/netchecker/netchecker-roles.yml'
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:58:50 ctl01 salt-minion[4526]: New file
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/addons/netchecker/netchecker-roles.yml] at time 13:58:50.308642 duration_in_ms=32.192
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/addons/prometheus/prometheus-roles.yml] at time 13:58:50.308931
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/kubernetes/addons/prometheus/prometheus-roles.yml]
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/kube-addons/prometheus/prometheus-roles.yml'
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:58:50 ctl01 salt-minion[4526]: New file
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/addons/prometheus/prometheus-roles.yml] at time 13:58:50.339536 duration_in_ms=30.604
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/addons/coredns] at time 13:58:50.339772
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Executing state file.absent for [/etc/kubernetes/addons/coredns]
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] File /etc/kubernetes/addons/coredns is not present
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/addons/coredns] at time 13:58:50.340597 duration_in_ms=0.825
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Running state [kubectl -n kube-system delete svc coredns > /dev/null || echo "coredns is absent. OK" && true] at time 13:58:50.341665
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Executing state cmd.run for [kubectl -n kube-system delete svc coredns > /dev/null || echo "coredns is absent. OK" && true]
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Executing command 'kubectl -n kube-system delete svc coredns > /dev/null || echo "coredns is absent. OK" && true' in directory '/root'
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] {'pid': 5380, 'retcode': 0, 'stderr': '/bin/sh: 1: kubectl: not found', 'stdout': 'coredns is absent. OK'}
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Completed state [kubectl -n kube-system delete svc coredns > /dev/null || echo "coredns is absent. OK" && true] at time 13:58:50.361642 duration_in_ms=19.977
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/addons/dns/kubedns-svc.yaml] at time 13:58:50.362227
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/kubernetes/addons/dns/kubedns-svc.yaml]
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/kube-addons/dns/kubedns-svc.yaml'
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:58:50 ctl01 salt-minion[4526]: New file
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/addons/dns/kubedns-svc.yaml] at time 13:58:50.477959 duration_in_ms=115.73
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/addons/dns/kubedns-rc.yaml] at time 13:58:50.478530
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/kubernetes/addons/dns/kubedns-rc.yaml]
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/kube-addons/dns/kubedns-rc.yaml'
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:58:50 ctl01 salt-minion[4526]: New file
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/addons/dns/kubedns-rc.yaml] at time 13:58:50.610002 duration_in_ms=131.47
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/addons/dns/kubedns-sa.yaml] at time 13:58:50.610485
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/kubernetes/addons/dns/kubedns-sa.yaml]
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/kube-addons/dns/kubedns-sa.yaml'
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:58:50 ctl01 salt-minion[4526]: New file
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/addons/dns/kubedns-sa.yaml] at time 13:58:50.644194 duration_in_ms=33.708
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/addons/dns/kubedns-autoscaler.yaml] at time 13:58:50.644450
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/kubernetes/addons/dns/kubedns-autoscaler.yaml]
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/kube-addons/dns/kubedns-autoscaler.yaml'
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:58:50 ctl01 salt-minion[4526]: New file
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/addons/dns/kubedns-autoscaler.yaml] at time 13:58:50.758002 duration_in_ms=113.551
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/addons/dns/kubedns-autoscaler-rbac.yaml] at time 13:58:50.758438
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/kubernetes/addons/dns/kubedns-autoscaler-rbac.yaml]
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/kube-addons/dns/kubedns-autoscaler-rbac.yaml'
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:58:50 ctl01 salt-minion[4526]: New file
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/addons/dns/kubedns-autoscaler-rbac.yaml] at time 13:58:50.790659 duration_in_ms=32.221
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/addons/dns/kubedns-clusterrole.yaml] at time 13:58:50.790897
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/kubernetes/addons/dns/kubedns-clusterrole.yaml]
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/kube-addons/dns/kubedns-clusterrole.yaml'
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:58:50 ctl01 salt-minion[4526]: New file
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/addons/dns/kubedns-clusterrole.yaml] at time 13:58:50.822014 duration_in_ms=31.116
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/addons/metrics-server] at time 13:58:50.822243
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Executing state file.absent for [/etc/kubernetes/addons/metrics-server]
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] File /etc/kubernetes/addons/metrics-server is not present
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/addons/metrics-server] at time 13:58:50.823010 duration_in_ms=0.767
Oct 26 13:58:50 ctl01 salt-minion[4526]: [INFO    ] Returning information for job: 20191026135842743315
Oct 26 13:58:51 ctl01 salt-minion[4526]: [INFO    ] User sudo_ubuntu Executing command state.sls with jid 20191026135851446315
Oct 26 13:58:51 ctl01 salt-minion[4526]: [INFO    ] Starting a new job with PID 5418
Oct 26 13:58:52 ctl01 salt-minion[4526]: [INFO    ] Loading fresh modules for state activity
Oct 26 13:58:52 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/pool/init.sls'
Oct 26 13:58:52 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:58:52 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:58:52 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:58:52 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:58:52 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/pool/calico.sls'
Oct 26 13:58:52 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:58:52 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:58:52 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:58:52 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:58:52 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/pool/service.sls'
Oct 26 13:58:52 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:58:52 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:58:52 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:58:52 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:58:53 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/_common.sls'
Oct 26 13:58:53 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:58:53 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:58:53 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/pool/kube-proxy.sls'
Oct 26 13:58:53 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:58:53 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:58:53 ctl01 salt-minion[4526]: [INFO    ] Running state [/usr/bin/calicoctl] at time 13:58:53.318196
Oct 26 13:58:53 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/usr/bin/calicoctl]
Oct 26 13:58:54 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:58:54 ctl01 salt-minion[4526]: New file
Oct 26 13:58:54 ctl01 salt-minion[4526]: [INFO    ] Completed state [/usr/bin/calicoctl] at time 13:58:54.678578 duration_in_ms=1360.383
Oct 26 13:58:54 ctl01 salt-minion[4526]: [INFO    ] Running state [/usr/bin/birdcl] at time 13:58:54.679042
Oct 26 13:58:54 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/usr/bin/birdcl]
Oct 26 13:58:55 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:58:55 ctl01 salt-minion[4526]: New file
Oct 26 13:58:55 ctl01 salt-minion[4526]: [INFO    ] Completed state [/usr/bin/birdcl] at time 13:58:55.017431 duration_in_ms=338.378
Oct 26 13:58:55 ctl01 salt-minion[4526]: [INFO    ] Running state [/opt/cni/bin/calico] at time 13:58:55.018147
Oct 26 13:58:55 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/opt/cni/bin/calico]
Oct 26 13:58:56 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:58:56 ctl01 salt-minion[4526]: New file
Oct 26 13:58:56 ctl01 salt-minion[4526]: [INFO    ] Completed state [/opt/cni/bin/calico] at time 13:58:56.161941 duration_in_ms=1143.792
Oct 26 13:58:56 ctl01 salt-minion[4526]: [INFO    ] Running state [/opt/cni/bin/calico-ipam] at time 13:58:56.162353
Oct 26 13:58:56 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/opt/cni/bin/calico-ipam]
Oct 26 13:58:57 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:58:57 ctl01 salt-minion[4526]: New file
Oct 26 13:58:57 ctl01 salt-minion[4526]: [INFO    ] Completed state [/opt/cni/bin/calico-ipam] at time 13:58:57.196769 duration_in_ms=1034.413
Oct 26 13:58:57 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/cni/net.d/10-calico.conf] at time 13:58:57.197103
Oct 26 13:58:57 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/cni/net.d/10-calico.conf]
Oct 26 13:58:57 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/calico/calico.conf'
Oct 26 13:58:57 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:58:57 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:58:57 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:58:57 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:58:57 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:58:57 ctl01 salt-minion[4526]: New file
Oct 26 13:58:57 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/cni/net.d/10-calico.conf] at time 13:58:57.339796 duration_in_ms=142.691
Oct 26 13:58:57 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/calico/network-environment] at time 13:58:57.340277
Oct 26 13:58:57 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/calico/network-environment]
Oct 26 13:58:57 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/calico/network-environment.pool'
Oct 26 13:58:57 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:58:57 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:58:57 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:58:57 ctl01 salt-minion[4526]: New file
Oct 26 13:58:57 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/calico/network-environment] at time 13:58:57.438868 duration_in_ms=98.59
Oct 26 13:58:57 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/calico/calicoctl.cfg] at time 13:58:57.439230
Oct 26 13:58:57 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/calico/calicoctl.cfg]
Oct 26 13:58:57 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/calico/calicoctl.cfg.pool'
Oct 26 13:58:57 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:58:57 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:58:57 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:58:57 ctl01 salt-minion[4526]: New file
Oct 26 13:58:57 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/calico/calicoctl.cfg] at time 13:58:57.540732 duration_in_ms=101.5
Oct 26 13:58:58 ctl01 salt-minion[4526]: [INFO    ] Running state [containerd=1.2.5-3~u16.04+mcp] at time 13:58:58.223456
Oct 26 13:58:58 ctl01 salt-minion[4526]: [INFO    ] Executing state pkg.installed for [containerd=1.2.5-3~u16.04+mcp]
Oct 26 13:58:58 ctl01 salt-minion[4526]: [INFO    ] Executing command ['dpkg-query', '--showformat', '${Status} ${Package} ${Version} ${Architecture}', '-W'] in directory '/root'
Oct 26 13:58:58 ctl01 salt-minion[4526]: [INFO    ] Executing command ['apt-cache', '-q', 'policy', 'containerd=1.2.5-3~u16.04+mcp'] in directory '/root'
Oct 26 13:58:58 ctl01 salt-minion[4526]: [INFO    ] Executing command ['apt-get', '-q', 'update'] in directory '/root'
Oct 26 13:59:01 ctl01 salt-minion[4526]: [INFO    ] Executing command ['dpkg', '--get-selections', '*'] in directory '/root'
Oct 26 13:59:01 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemd-run', '--scope', 'apt-get', '-q', '-y', '-o', 'DPkg::Options::=--force-confold', '-o', 'DPkg::Options::=--force-confdef', 'install', 'containerd=1.2.5-3~u16.04+mcp'] in directory '/root'
Oct 26 13:59:01 ctl01 systemd[1]: Started /usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold -o DPkg::Options::=--force-confdef install containerd=1.2.5-3~u16.04+mcp.
Oct 26 13:59:06 ctl01 salt-minion[4526]: [INFO    ] User sudo_ubuntu Executing command saltutil.find_job with jid 20191026135906571083
Oct 26 13:59:06 ctl01 salt-minion[4526]: [INFO    ] Starting a new job with PID 5985
Oct 26 13:59:06 ctl01 salt-minion[4526]: [INFO    ] Returning information for job: 20191026135906571083
Oct 26 13:59:13 ctl01 systemd[1]: Reloading.
Oct 26 13:59:13 ctl01 systemd[1]: Started ACPI event daemon.
Oct 26 13:59:13 ctl01 systemd[1]: Reloading.
Oct 26 13:59:13 ctl01 systemd[1]: Started ACPI event daemon.
Oct 26 13:59:13 ctl01 systemd[1]: Starting containerd container runtime...
Oct 26 13:59:13 ctl01 systemd[1]: Started containerd container runtime.
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.590641821Z" level=info msg="starting containerd" revision= version=1.2.5-3
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.591544498Z" level=info msg="loading plugin "io.containerd.content.v1.content"..." type=io.containerd.content.v1
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.591896938Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.btrfs"..." type=io.containerd.snapshotter.v1
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.592323452Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.btrfs" error="path /var/lib/containerd/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter"
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.592639063Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.aufs"..." type=io.containerd.snapshotter.v1
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.611105522Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.native"..." type=io.containerd.snapshotter.v1
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.611624012Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.overlayfs"..." type=io.containerd.snapshotter.v1
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.612017081Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.zfs"..." type=io.containerd.snapshotter.v1
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.612468888Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.zfs" error="path /var/lib/containerd/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter"
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.612707401Z" level=info msg="loading plugin "io.containerd.metadata.v1.bolt"..." type=io.containerd.metadata.v1
Oct 26 13:59:13 ctl01 kernel: [  151.297396] aufs 4.15-20180219
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.612970752Z" level=warning msg="could not use snapshotter btrfs in metadata plugin" error="path /var/lib/containerd/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter"
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.613256943Z" level=warning msg="could not use snapshotter zfs in metadata plugin" error="path /var/lib/containerd/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter"
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.629856116Z" level=info msg="loading plugin "io.containerd.differ.v1.walking"..." type=io.containerd.differ.v1
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.630370860Z" level=info msg="loading plugin "io.containerd.gc.v1.scheduler"..." type=io.containerd.gc.v1
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.630630393Z" level=info msg="loading plugin "io.containerd.service.v1.containers-service"..." type=io.containerd.service.v1
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.630862903Z" level=info msg="loading plugin "io.containerd.service.v1.content-service"..." type=io.containerd.service.v1
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.631076660Z" level=info msg="loading plugin "io.containerd.service.v1.diff-service"..." type=io.containerd.service.v1
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.631301142Z" level=info msg="loading plugin "io.containerd.service.v1.images-service"..." type=io.containerd.service.v1
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.631505650Z" level=info msg="loading plugin "io.containerd.service.v1.leases-service"..." type=io.containerd.service.v1
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.631717616Z" level=info msg="loading plugin "io.containerd.service.v1.namespaces-service"..." type=io.containerd.service.v1
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.631919518Z" level=info msg="loading plugin "io.containerd.service.v1.snapshots-service"..." type=io.containerd.service.v1
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.632118396Z" level=info msg="loading plugin "io.containerd.runtime.v1.linux"..." type=io.containerd.runtime.v1
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.632463797Z" level=info msg="loading plugin "io.containerd.runtime.v2.task"..." type=io.containerd.runtime.v2
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.632757575Z" level=info msg="loading plugin "io.containerd.monitor.v1.cgroups"..." type=io.containerd.monitor.v1
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.633444968Z" level=info msg="loading plugin "io.containerd.service.v1.tasks-service"..." type=io.containerd.service.v1
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.633776836Z" level=info msg="loading plugin "io.containerd.internal.v1.restart"..." type=io.containerd.internal.v1
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.634102832Z" level=info msg="loading plugin "io.containerd.grpc.v1.containers"..." type=io.containerd.grpc.v1
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.634326350Z" level=info msg="loading plugin "io.containerd.grpc.v1.content"..." type=io.containerd.grpc.v1
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.634535783Z" level=info msg="loading plugin "io.containerd.grpc.v1.diff"..." type=io.containerd.grpc.v1
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.634748108Z" level=info msg="loading plugin "io.containerd.grpc.v1.events"..." type=io.containerd.grpc.v1
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.634947926Z" level=info msg="loading plugin "io.containerd.grpc.v1.healthcheck"..." type=io.containerd.grpc.v1
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.635152329Z" level=info msg="loading plugin "io.containerd.grpc.v1.images"..." type=io.containerd.grpc.v1
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.635351291Z" level=info msg="loading plugin "io.containerd.grpc.v1.leases"..." type=io.containerd.grpc.v1
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.635553560Z" level=info msg="loading plugin "io.containerd.grpc.v1.namespaces"..." type=io.containerd.grpc.v1
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.635749715Z" level=info msg="loading plugin "io.containerd.internal.v1.opt"..." type=io.containerd.internal.v1
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.636099537Z" level=info msg="loading plugin "io.containerd.grpc.v1.snapshots"..." type=io.containerd.grpc.v1
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.636307533Z" level=info msg="loading plugin "io.containerd.grpc.v1.tasks"..." type=io.containerd.grpc.v1
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.636507324Z" level=info msg="loading plugin "io.containerd.grpc.v1.version"..." type=io.containerd.grpc.v1
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.636728647Z" level=info msg="loading plugin "io.containerd.grpc.v1.cri"..." type=io.containerd.grpc.v1
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.637015271Z" level=info msg="Start cri plugin with config {PluginConfig:{ContainerdConfig:{Snapshotter:overlayfs DefaultRuntime:{Type:io.containerd.runtime.v1.linux Engine: Root: Options:<nil>} UntrustedWorkloadRuntime:{Type: Engine: Root: Options:<nil>} Runtimes:map[] NoPivot:false} CniConfig:{NetworkPluginBinDir:/opt/cni/bin NetworkPluginConfDir:/etc/cni/net.d NetworkPluginConfTemplate:} Registry:{Mirrors:map[docker.io:{Endpoints:[https://registry-1.docker.io]}] Auths:map[]} StreamServerAddress:127.0.0.1 StreamServerPort:0 EnableSelinux:false SandboxImage:k8s.gcr.io/pause:3.1 StatsCollectPeriod:10 SystemdCgroup:false EnableTLSStreaming:false X509KeyPairStreaming:{TLSCertFile: TLSKeyFile:} MaxContainerLogLineSize:16384} ContainerdRootDir:/var/lib/containerd ContainerdEndpoint:/run/containerd/containerd.sock RootDir:/var/lib/containerd/io.containerd.grpc.v1.cri StateDir:/run/containerd/io.containerd.grpc.v1.cri}"
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.638370154Z" level=info msg="Connect containerd service"
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.638757959Z" level=info msg="Get image filesystem path "/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs""
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.639717983Z" level=info msg="loading plugin "io.containerd.grpc.v1.introspection"..." type=io.containerd.grpc.v1
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.640106060Z" level=info msg=serving... address="/run/containerd/containerd.sock"
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.640175964Z" level=info msg="containerd successfully booted in 0.050171s"
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.641029196Z" level=info msg="Start subscribing containerd event"
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.641323718Z" level=info msg="Start recovering state"
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.641751713Z" level=info msg="Start event monitor"
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.642042470Z" level=info msg="Start snapshots syncer"
Oct 26 13:59:13 ctl01 containerd[6377]: time="2019-10-26T13:59:13.642297183Z" level=info msg="Start streaming server"
Oct 26 13:59:14 ctl01 salt-minion[4526]: [INFO    ] Executing command ['dpkg-query', '--showformat', '${Status} ${Package} ${Version} ${Architecture}', '-W'] in directory '/root'
Oct 26 13:59:14 ctl01 salt-minion[4526]: [INFO    ] Made the following changes:
Oct 26 13:59:14 ctl01 salt-minion[4526]: 'containerd' changed from 'absent' to '1.2.5-3~u16.04+mcp'
Oct 26 13:59:14 ctl01 salt-minion[4526]: 'runc' changed from 'absent' to '1.0.0~rc7+git20190403.029124da-0ubuntu1~16.04.4'
Oct 26 13:59:14 ctl01 salt-minion[4526]: [INFO    ] Loading fresh modules for state activity
Oct 26 13:59:14 ctl01 salt-minion[4526]: [INFO    ] Completed state [containerd=1.2.5-3~u16.04+mcp] at time 13:59:14.795018 duration_in_ms=16571.562
Oct 26 13:59:14 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/containerd/config.toml] at time 13:59:14.798851
Oct 26 13:59:14 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/containerd/config.toml]
Oct 26 13:59:14 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/containerd/config.toml'
Oct 26 13:59:14 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:59:14 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:59:14 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:59:14 ctl01 salt-minion[4526]: New file
Oct 26 13:59:14 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/containerd/config.toml] at time 13:59:14.905272 duration_in_ms=106.42
Oct 26 13:59:15 ctl01 salt-minion[4526]: [INFO    ] Running state [containerd] at time 13:59:15.369498
Oct 26 13:59:15 ctl01 salt-minion[4526]: [INFO    ] Executing state service.running for [containerd]
Oct 26 13:59:15 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'status', 'containerd.service', '-n', '0'] in directory '/root'
Oct 26 13:59:15 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-active', 'containerd.service'] in directory '/root'
Oct 26 13:59:15 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-enabled', 'containerd.service'] in directory '/root'
Oct 26 13:59:15 ctl01 salt-minion[4526]: [INFO    ] The service containerd is already running
Oct 26 13:59:15 ctl01 salt-minion[4526]: [INFO    ] Completed state [containerd] at time 13:59:15.416720 duration_in_ms=47.222
Oct 26 13:59:15 ctl01 salt-minion[4526]: [INFO    ] Running state [containerd] at time 13:59:15.417204
Oct 26 13:59:15 ctl01 salt-minion[4526]: [INFO    ] Executing state service.mod_watch for [containerd]
Oct 26 13:59:15 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-active', 'containerd.service'] in directory '/root'
Oct 26 13:59:15 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemd-run', '--scope', 'systemctl', 'restart', 'containerd.service'] in directory '/root'
Oct 26 13:59:15 ctl01 systemd[1]: Started /bin/systemctl restart containerd.service.
Oct 26 13:59:15 ctl01 systemd[1]: Stopping containerd container runtime...
Oct 26 13:59:15 ctl01 containerd[6377]: time="2019-10-26T13:59:15.453942769Z" level=info msg="Stop CRI service"
Oct 26 13:59:15 ctl01 systemd[1]: Stopped containerd container runtime.
Oct 26 13:59:15 ctl01 systemd[1]: Starting containerd container runtime...
Oct 26 13:59:15 ctl01 systemd[1]: Started containerd container runtime.
Oct 26 13:59:15 ctl01 salt-minion[4526]: [INFO    ] {'containerd': True}
Oct 26 13:59:15 ctl01 salt-minion[4526]: [INFO    ] Completed state [containerd] at time 13:59:15.464910 duration_in_ms=47.704
Oct 26 13:59:15 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/systemd/system/calico-node.service] at time 13:59:15.466020
Oct 26 13:59:15 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/systemd/system/calico-node.service]
Oct 26 13:59:15 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/calico/calico-node.service.ctr'
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.494923494Z" level=info msg="starting containerd" revision= version=1.2.5-3
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.495409858Z" level=info msg="loading plugin "io.containerd.content.v1.content"..." type=io.containerd.content.v1
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.495475565Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.btrfs"..." type=io.containerd.snapshotter.v1
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.495683623Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.btrfs" error="path /var/lib/containerd/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter"
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.495777930Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.aufs"..." type=io.containerd.snapshotter.v1
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.503092157Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.native"..." type=io.containerd.snapshotter.v1
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.503153658Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.overlayfs"..." type=io.containerd.snapshotter.v1
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.503274644Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.zfs"..." type=io.containerd.snapshotter.v1
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.503493330Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.zfs" error="path /var/lib/containerd/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter"
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.503524338Z" level=info msg="loading plugin "io.containerd.metadata.v1.bolt"..." type=io.containerd.metadata.v1
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.503549213Z" level=warning msg="could not use snapshotter zfs in metadata plugin" error="path /var/lib/containerd/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter"
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.503560440Z" level=warning msg="could not use snapshotter btrfs in metadata plugin" error="path /var/lib/containerd/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter"
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.503695577Z" level=info msg="loading plugin "io.containerd.differ.v1.walking"..." type=io.containerd.differ.v1
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.503723957Z" level=info msg="loading plugin "io.containerd.gc.v1.scheduler"..." type=io.containerd.gc.v1
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.503781289Z" level=info msg="loading plugin "io.containerd.service.v1.containers-service"..." type=io.containerd.service.v1
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.503815371Z" level=info msg="loading plugin "io.containerd.service.v1.content-service"..." type=io.containerd.service.v1
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.503853253Z" level=info msg="loading plugin "io.containerd.service.v1.diff-service"..." type=io.containerd.service.v1
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.503873804Z" level=info msg="loading plugin "io.containerd.service.v1.images-service"..." type=io.containerd.service.v1
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.503892383Z" level=info msg="loading plugin "io.containerd.service.v1.leases-service"..." type=io.containerd.service.v1
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.503910878Z" level=info msg="loading plugin "io.containerd.service.v1.namespaces-service"..." type=io.containerd.service.v1
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.503929844Z" level=info msg="loading plugin "io.containerd.service.v1.snapshots-service"..." type=io.containerd.service.v1
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.503949008Z" level=info msg="loading plugin "io.containerd.runtime.v1.linux"..." type=io.containerd.runtime.v1
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.504007065Z" level=info msg="loading plugin "io.containerd.runtime.v2.task"..." type=io.containerd.runtime.v2
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.504072528Z" level=info msg="loading plugin "io.containerd.monitor.v1.cgroups"..." type=io.containerd.monitor.v1
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.504574947Z" level=info msg="loading plugin "io.containerd.service.v1.tasks-service"..." type=io.containerd.service.v1
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.504604899Z" level=info msg="loading plugin "io.containerd.internal.v1.restart"..." type=io.containerd.internal.v1
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.504662232Z" level=info msg="loading plugin "io.containerd.grpc.v1.containers"..." type=io.containerd.grpc.v1
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.504679148Z" level=info msg="loading plugin "io.containerd.grpc.v1.content"..." type=io.containerd.grpc.v1
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.504692386Z" level=info msg="loading plugin "io.containerd.grpc.v1.diff"..." type=io.containerd.grpc.v1
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.504705115Z" level=info msg="loading plugin "io.containerd.grpc.v1.events"..." type=io.containerd.grpc.v1
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.504717525Z" level=info msg="loading plugin "io.containerd.grpc.v1.healthcheck"..." type=io.containerd.grpc.v1
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.504730133Z" level=info msg="loading plugin "io.containerd.grpc.v1.images"..." type=io.containerd.grpc.v1
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.504742502Z" level=info msg="loading plugin "io.containerd.grpc.v1.leases"..." type=io.containerd.grpc.v1
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.504755216Z" level=info msg="loading plugin "io.containerd.grpc.v1.namespaces"..." type=io.containerd.grpc.v1
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.504768499Z" level=info msg="loading plugin "io.containerd.internal.v1.opt"..." type=io.containerd.internal.v1
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.504845695Z" level=info msg="loading plugin "io.containerd.grpc.v1.snapshots"..." type=io.containerd.grpc.v1
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.504872599Z" level=info msg="loading plugin "io.containerd.grpc.v1.tasks"..." type=io.containerd.grpc.v1
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.504894115Z" level=info msg="loading plugin "io.containerd.grpc.v1.version"..." type=io.containerd.grpc.v1
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.504912249Z" level=info msg="loading plugin "io.containerd.grpc.v1.cri"..." type=io.containerd.grpc.v1
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.505463627Z" level=info msg="Start cri plugin with config {PluginConfig:{ContainerdConfig:{Snapshotter:overlayfs DefaultRuntime:{Type:io.containerd.runtime.v1.linux Engine: Root: Options:<nil>} UntrustedWorkloadRuntime:{Type: Engine: Root: Options:<nil>} Runtimes:map[] NoPivot:false} CniConfig:{NetworkPluginBinDir:/opt/cni/bin NetworkPluginConfDir:/etc/cni/net.d NetworkPluginConfTemplate:} Registry:{Mirrors:map[docker.io:{Endpoints:[https://registry-1.docker.io]}] Auths:map[]} StreamServerAddress:127.0.0.1 StreamServerPort:0 EnableSelinux:false SandboxImage:docker-prod-local.artifactory.mirantis.com/mirantis/kubernetes/pause-amd64:v1.13.5-3 StatsCollectPeriod:10 SystemdCgroup:false EnableTLSStreaming:false X509KeyPairStreaming:{TLSCertFile: TLSKeyFile:} MaxContainerLogLineSize:16384} ContainerdRootDir:/var/lib/containerd ContainerdEndpoint:/run/containerd/containerd.sock RootDir:/var/lib/containerd/io.containerd.grpc.v1.cri StateDir:/run/containerd/io.containerd.grpc.v1.cri}"
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.505550552Z" level=info msg="Connect containerd service"
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.505707424Z" level=info msg="Get image filesystem path "/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs""
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.506196785Z" level=info msg="loading plugin "io.containerd.grpc.v1.introspection"..." type=io.containerd.grpc.v1
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.506419449Z" level=info msg=serving... address="/run/containerd/containerd.sock"
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.506436403Z" level=info msg="containerd successfully booted in 0.011934s"
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.506460166Z" level=info msg="Start subscribing containerd event"
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.506560846Z" level=info msg="Start recovering state"
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.506967992Z" level=info msg="Start event monitor"
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.507008072Z" level=info msg="Start snapshots syncer"
Oct 26 13:59:15 ctl01 containerd[6508]: time="2019-10-26T13:59:15.507019817Z" level=info msg="Start streaming server"
Oct 26 13:59:15 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:59:15 ctl01 salt-minion[4526]: New file
Oct 26 13:59:15 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/systemd/system/calico-node.service] at time 13:59:15.509454 duration_in_ms=43.434
Oct 26 13:59:15 ctl01 salt-minion[4526]: [INFO    ] Running state [/var/lib/calico] at time 13:59:15.509752
Oct 26 13:59:15 ctl01 salt-minion[4526]: [INFO    ] Executing state file.directory for [/var/lib/calico]
Oct 26 13:59:15 ctl01 salt-minion[4526]: [INFO    ] {'/var/lib/calico': 'New Dir'}
Oct 26 13:59:15 ctl01 salt-minion[4526]: [INFO    ] Completed state [/var/lib/calico] at time 13:59:15.512048 duration_in_ms=2.297
Oct 26 13:59:15 ctl01 salt-minion[4526]: [INFO    ] Running state [/var/log/calico] at time 13:59:15.512319
Oct 26 13:59:15 ctl01 salt-minion[4526]: [INFO    ] Executing state file.directory for [/var/log/calico]
Oct 26 13:59:15 ctl01 salt-minion[4526]: [INFO    ] {'/var/log/calico': 'New Dir'}
Oct 26 13:59:15 ctl01 salt-minion[4526]: [INFO    ] Completed state [/var/log/calico] at time 13:59:15.514028 duration_in_ms=1.708
Oct 26 13:59:15 ctl01 salt-minion[4526]: [INFO    ] Running state [calico-node] at time 13:59:15.516232
Oct 26 13:59:15 ctl01 salt-minion[4526]: [INFO    ] Executing state service.running for [calico-node]
Oct 26 13:59:15 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'status', 'calico-node.service', '-n', '0'] in directory '/root'
Oct 26 13:59:15 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-active', 'calico-node.service'] in directory '/root'
Oct 26 13:59:15 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-enabled', 'calico-node.service'] in directory '/root'
Oct 26 13:59:15 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemd-run', '--scope', 'systemctl', 'start', 'calico-node.service'] in directory '/root'
Oct 26 13:59:15 ctl01 systemd[1]: Started /bin/systemctl start calico-node.service.
Oct 26 13:59:15 ctl01 systemd[1]: Starting calico-node...
Oct 26 13:59:15 ctl01 ctr[6560]: ctr: container "calico-node" in namespace "default": not found
Oct 26 13:59:15 ctl01 ctr[6580]: time="2019-10-26T13:59:15Z" level=error msg="failed to delete container "calico-node"" error="container "calico-node" in namespace "default": not found"
Oct 26 13:59:15 ctl01 ctr[6580]: ctr: container "calico-node" in namespace "default": not found
Oct 26 13:59:15 ctl01 ctr[6590]: docker-prod-local.artifactory.mirantis.com/mirantis/projectcalico/calico/node:v3.3.2: resolving      |#033[32m#033[0m--------------------------------------|
Oct 26 13:59:15 ctl01 ctr[6590]: elapsed: 0.1 s                                                                        total:   0.0 B (0.0 B/s)
Oct 26 13:59:15 ctl01 ctr[6590]: docker-prod-local.artifactory.mirantis.com/mirantis/projectcalico/calico/node:v3.3.2: resolving      |#033[32m#033[0m--------------------------------------|
Oct 26 13:59:15 ctl01 ctr[6590]: elapsed: 0.2 s                                                                        total:   0.0 B (0.0 B/s)
Oct 26 13:59:16 ctl01 ctr[6590]: docker-prod-local.artifactory.mirantis.com/mirantis/projectcalico/calico/node:v3.3.2: resolving      |#033[32m#033[0m--------------------------------------|
Oct 26 13:59:16 ctl01 ctr[6590]: elapsed: 0.3 s                                                                        total:   0.0 B (0.0 B/s)
Oct 26 13:59:16 ctl01 ctr[6590]: docker-prod-local.artifactory.mirantis.com/mirantis/projectcalico/calico/node:v3.3.2: resolving      |#033[32m#033[0m--------------------------------------|
Oct 26 13:59:16 ctl01 ctr[6590]: elapsed: 0.4 s                                                                        total:   0.0 B (0.0 B/s)
Oct 26 13:59:16 ctl01 ctr[6590]: docker-prod-local.artifactory.mirantis.com/mirantis/projectcalico/calico/node:v3.3.2: resolving      |#033[32m#033[0m--------------------------------------|
Oct 26 13:59:16 ctl01 ctr[6590]: elapsed: 0.5 s                                                                        total:   0.0 B (0.0 B/s)
Oct 26 13:59:16 ctl01 ctr[6590]: docker-prod-local.artifactory.mirantis.com/mirantis/projectcalico/calico/node:v3.3.2: resolved       |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:16 ctl01 ctr[6590]: manifest-sha256:4b3e3750deeb97cf6f68e5d021f60891a0562f7412efdb545599e6ea505eaf18:     downloading    |#033[32m#033[0m--------------------------------------|    0.0 B/1.3 KiB
Oct 26 13:59:16 ctl01 ctr[6590]: elapsed: 0.6 s                                                                        total:   0.0 B (0.0 B/s)
Oct 26 13:59:16 ctl01 ctr[6590]: docker-prod-local.artifactory.mirantis.com/mirantis/projectcalico/calico/node:v3.3.2: resolved       |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:16 ctl01 ctr[6590]: manifest-sha256:4b3e3750deeb97cf6f68e5d021f60891a0562f7412efdb545599e6ea505eaf18:     downloading    |#033[32m#033[0m--------------------------------------|    0.0 B/1.3 KiB
Oct 26 13:59:16 ctl01 ctr[6590]: elapsed: 0.7 s                                                                        total:   0.0 B (0.0 B/s)
Oct 26 13:59:16 ctl01 ctr[6590]: docker-prod-local.artifactory.mirantis.com/mirantis/projectcalico/calico/node:v3.3.2: resolved       |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:16 ctl01 ctr[6590]: manifest-sha256:4b3e3750deeb97cf6f68e5d021f60891a0562f7412efdb545599e6ea505eaf18:     downloading    |#033[32m#033[0m--------------------------------------|    0.0 B/1.3 KiB
Oct 26 13:59:16 ctl01 ctr[6590]: elapsed: 0.8 s                                                                        total:   0.0 B (0.0 B/s)
Oct 26 13:59:16 ctl01 ctr[6590]: docker-prod-local.artifactory.mirantis.com/mirantis/projectcalico/calico/node:v3.3.2: resolved       |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:16 ctl01 ctr[6590]: manifest-sha256:4b3e3750deeb97cf6f68e5d021f60891a0562f7412efdb545599e6ea505eaf18:     done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:16 ctl01 ctr[6590]: config-sha256:4e9be81e3a5948d40df6358fdae2cc0dde85a0085723666c50ee7d15427a9b48:       downloading    |#033[32m#033[0m--------------------------------------|    0.0 B/3.2 KiB
Oct 26 13:59:16 ctl01 ctr[6590]: layer-sha256:3f90cdf570685ae358ac0456a9d07fffc96858427c71d754f82e222d96f1c683:        downloading    |#033[32m#033[0m--------------------------------------|    0.0 B/1.9 MiB
Oct 26 13:59:16 ctl01 ctr[6590]: layer-sha256:37159c5154b88277f12fe9aa20d728ca5c92fd38e6e707660ee27eef281de923:        downloading    |#033[32m#033[0m--------------------------------------|    0.0 B/47.1 KiB
Oct 26 13:59:16 ctl01 ctr[6590]: layer-sha256:b788fa0813576d69f1efd12e893b97db7d007b5b93ca1cce6663f96ba1ab6488:        downloading    |#033[32m#033[0m--------------------------------------|    0.0 B/1.8 MiB
Oct 26 13:59:16 ctl01 ctr[6590]: layer-sha256:4fe2ade4980c2dda4fc95858ebb981489baec8c1e4bd282ab1c3560be8ff9bde:        downloading    |#033[32m#033[0m--------------------------------------|    0.0 B/2.1 MiB
Oct 26 13:59:16 ctl01 ctr[6590]: layer-sha256:798a8ef97f6bac88a42e34bf15d8e412242025372c4f567590579ba37d381b2c:        downloading    |#033[32m#033[0m--------------------------------------|    0.0 B/13.7 MiB
Oct 26 13:59:16 ctl01 ctr[6590]: elapsed: 0.9 s                                                                        total:  1.3 Ki (1.5 KiB/s)
Oct 26 13:59:16 ctl01 ctr[6590]: docker-prod-local.artifactory.mirantis.com/mirantis/projectcalico/calico/node:v3.3.2: resolved       |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:16 ctl01 ctr[6590]: manifest-sha256:4b3e3750deeb97cf6f68e5d021f60891a0562f7412efdb545599e6ea505eaf18:     done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:16 ctl01 ctr[6590]: config-sha256:4e9be81e3a5948d40df6358fdae2cc0dde85a0085723666c50ee7d15427a9b48:       downloading    |#033[32m#033[0m--------------------------------------|    0.0 B/3.2 KiB
Oct 26 13:59:16 ctl01 ctr[6590]: layer-sha256:3f90cdf570685ae358ac0456a9d07fffc96858427c71d754f82e222d96f1c683:        downloading    |#033[32m#033[0m--------------------------------------|    0.0 B/1.9 MiB
Oct 26 13:59:16 ctl01 ctr[6590]: layer-sha256:37159c5154b88277f12fe9aa20d728ca5c92fd38e6e707660ee27eef281de923:        downloading    |#033[32m#033[0m--------------------------------------|    0.0 B/47.1 KiB
Oct 26 13:59:16 ctl01 ctr[6590]: layer-sha256:b788fa0813576d69f1efd12e893b97db7d007b5b93ca1cce6663f96ba1ab6488:        downloading    |#033[32m#033[0m--------------------------------------|    0.0 B/1.8 MiB
Oct 26 13:59:16 ctl01 ctr[6590]: layer-sha256:4fe2ade4980c2dda4fc95858ebb981489baec8c1e4bd282ab1c3560be8ff9bde:        downloading    |#033[32m#033[0m--------------------------------------|    0.0 B/2.1 MiB
Oct 26 13:59:16 ctl01 ctr[6590]: layer-sha256:798a8ef97f6bac88a42e34bf15d8e412242025372c4f567590579ba37d381b2c:        downloading    |#033[32m#033[0m--------------------------------------|    0.0 B/13.7 MiB
Oct 26 13:59:16 ctl01 ctr[6590]: elapsed: 1.0 s                                                                        total:  1.3 Ki (1.3 KiB/s)
Oct 26 13:59:16 ctl01 ctr[6590]: docker-prod-local.artifactory.mirantis.com/mirantis/projectcalico/calico/node:v3.3.2: resolved       |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:16 ctl01 ctr[6590]: manifest-sha256:4b3e3750deeb97cf6f68e5d021f60891a0562f7412efdb545599e6ea505eaf18:     done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:16 ctl01 ctr[6590]: config-sha256:4e9be81e3a5948d40df6358fdae2cc0dde85a0085723666c50ee7d15427a9b48:       done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:16 ctl01 ctr[6590]: layer-sha256:3f90cdf570685ae358ac0456a9d07fffc96858427c71d754f82e222d96f1c683:        downloading    |#033[32m#033[0m--------------------------------------| 31.1 KiB/1.9 MiB
Oct 26 13:59:16 ctl01 ctr[6590]: layer-sha256:37159c5154b88277f12fe9aa20d728ca5c92fd38e6e707660ee27eef281de923:        downloading    |#033[32m++++++++++++++++++#033[0m--------------------| 23.1 KiB/47.1 KiB
Oct 26 13:59:16 ctl01 ctr[6590]: layer-sha256:b788fa0813576d69f1efd12e893b97db7d007b5b93ca1cce6663f96ba1ab6488:        downloading    |#033[32m#033[0m--------------------------------------| 31.1 KiB/1.8 MiB
Oct 26 13:59:16 ctl01 ctr[6590]: layer-sha256:4fe2ade4980c2dda4fc95858ebb981489baec8c1e4bd282ab1c3560be8ff9bde:        downloading    |#033[32m#033[0m--------------------------------------| 39.1 KiB/2.1 MiB
Oct 26 13:59:16 ctl01 ctr[6590]: layer-sha256:798a8ef97f6bac88a42e34bf15d8e412242025372c4f567590579ba37d381b2c:        downloading    |#033[32m#033[0m--------------------------------------| 39.1 KiB/13.7 MiB
Oct 26 13:59:16 ctl01 ctr[6590]: elapsed: 1.1 s                                                                        total:  168.2  (152.7 KiB/s)
Oct 26 13:59:16 ctl01 ctr[6590]: docker-prod-local.artifactory.mirantis.com/mirantis/projectcalico/calico/node:v3.3.2: resolved       |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:16 ctl01 ctr[6590]: manifest-sha256:4b3e3750deeb97cf6f68e5d021f60891a0562f7412efdb545599e6ea505eaf18:     done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:16 ctl01 ctr[6590]: config-sha256:4e9be81e3a5948d40df6358fdae2cc0dde85a0085723666c50ee7d15427a9b48:       done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:16 ctl01 ctr[6590]: layer-sha256:3f90cdf570685ae358ac0456a9d07fffc96858427c71d754f82e222d96f1c683:        downloading    |#033[32m+++++++#033[0m-------------------------------| 383.2 Ki/1.9 MiB
Oct 26 13:59:16 ctl01 ctr[6590]: layer-sha256:37159c5154b88277f12fe9aa20d728ca5c92fd38e6e707660ee27eef281de923:        done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:16 ctl01 ctr[6590]: layer-sha256:b788fa0813576d69f1efd12e893b97db7d007b5b93ca1cce6663f96ba1ab6488:        downloading    |#033[32m++++++++#033[0m------------------------------| 407.1 Ki/1.8 MiB
Oct 26 13:59:16 ctl01 ctr[6590]: layer-sha256:4fe2ade4980c2dda4fc95858ebb981489baec8c1e4bd282ab1c3560be8ff9bde:        downloading    |#033[32m+++++++#033[0m-------------------------------| 431.2 Ki/2.1 MiB
Oct 26 13:59:16 ctl01 ctr[6590]: layer-sha256:798a8ef97f6bac88a42e34bf15d8e412242025372c4f567590579ba37d381b2c:        downloading    |#033[32m#033[0m--------------------------------------| 207.1 Ki/13.7 MiB
Oct 26 13:59:16 ctl01 ctr[6590]: elapsed: 1.2 s                                                                        total:  1.4 Mi (1.2 MiB/s)
Oct 26 13:59:17 ctl01 ctr[6590]: docker-prod-local.artifactory.mirantis.com/mirantis/projectcalico/calico/node:v3.3.2: resolved       |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: manifest-sha256:4b3e3750deeb97cf6f68e5d021f60891a0562f7412efdb545599e6ea505eaf18:     done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: config-sha256:4e9be81e3a5948d40df6358fdae2cc0dde85a0085723666c50ee7d15427a9b48:       done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:3f90cdf570685ae358ac0456a9d07fffc96858427c71d754f82e222d96f1c683:        downloading    |#033[32m+++++++++++++++++++++++++++++++#033[0m-------|  1.5 MiB/1.9 MiB
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:37159c5154b88277f12fe9aa20d728ca5c92fd38e6e707660ee27eef281de923:        done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:b788fa0813576d69f1efd12e893b97db7d007b5b93ca1cce6663f96ba1ab6488:        downloading    |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|  1.8 MiB/1.8 MiB
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:4fe2ade4980c2dda4fc95858ebb981489baec8c1e4bd282ab1c3560be8ff9bde:        downloading    |#033[32m+++++++++++++++++++++++++++++#033[0m---------|  1.6 MiB/2.1 MiB
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:798a8ef97f6bac88a42e34bf15d8e412242025372c4f567590579ba37d381b2c:        downloading    |#033[32m+++#033[0m-----------------------------------|  1.2 MiB/13.7 MiB
Oct 26 13:59:17 ctl01 ctr[6590]: elapsed: 1.3 s                                                                        total:  6.2 Mi (4.8 MiB/s)
Oct 26 13:59:17 ctl01 ctr[6590]: docker-prod-local.artifactory.mirantis.com/mirantis/projectcalico/calico/node:v3.3.2: resolved       |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: manifest-sha256:4b3e3750deeb97cf6f68e5d021f60891a0562f7412efdb545599e6ea505eaf18:     done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: config-sha256:4e9be81e3a5948d40df6358fdae2cc0dde85a0085723666c50ee7d15427a9b48:       done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:3f90cdf570685ae358ac0456a9d07fffc96858427c71d754f82e222d96f1c683:        done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:37159c5154b88277f12fe9aa20d728ca5c92fd38e6e707660ee27eef281de923:        done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:b788fa0813576d69f1efd12e893b97db7d007b5b93ca1cce6663f96ba1ab6488:        done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:4fe2ade4980c2dda4fc95858ebb981489baec8c1e4bd282ab1c3560be8ff9bde:        done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:798a8ef97f6bac88a42e34bf15d8e412242025372c4f567590579ba37d381b2c:        downloading    |#033[32m++++++++#033[0m------------------------------|  3.2 MiB/13.7 MiB
Oct 26 13:59:17 ctl01 ctr[6590]: elapsed: 1.4 s                                                                        total:  9.0 Mi (6.4 MiB/s)
Oct 26 13:59:17 ctl01 ctr[6590]: docker-prod-local.artifactory.mirantis.com/mirantis/projectcalico/calico/node:v3.3.2: resolved       |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: manifest-sha256:4b3e3750deeb97cf6f68e5d021f60891a0562f7412efdb545599e6ea505eaf18:     done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: config-sha256:4e9be81e3a5948d40df6358fdae2cc0dde85a0085723666c50ee7d15427a9b48:       done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:3f90cdf570685ae358ac0456a9d07fffc96858427c71d754f82e222d96f1c683:        done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:37159c5154b88277f12fe9aa20d728ca5c92fd38e6e707660ee27eef281de923:        done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:b788fa0813576d69f1efd12e893b97db7d007b5b93ca1cce6663f96ba1ab6488:        done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:4fe2ade4980c2dda4fc95858ebb981489baec8c1e4bd282ab1c3560be8ff9bde:        done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:798a8ef97f6bac88a42e34bf15d8e412242025372c4f567590579ba37d381b2c:        downloading    |#033[32m++++++++++++++#033[0m------------------------|  5.3 MiB/13.7 MiB
Oct 26 13:59:17 ctl01 ctr[6590]: elapsed: 1.5 s                                                                        total:  11.2 M (7.4 MiB/s)
Oct 26 13:59:17 ctl01 ctr[6590]: docker-prod-local.artifactory.mirantis.com/mirantis/projectcalico/calico/node:v3.3.2: resolved       |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: manifest-sha256:4b3e3750deeb97cf6f68e5d021f60891a0562f7412efdb545599e6ea505eaf18:     done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: config-sha256:4e9be81e3a5948d40df6358fdae2cc0dde85a0085723666c50ee7d15427a9b48:       done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:3f90cdf570685ae358ac0456a9d07fffc96858427c71d754f82e222d96f1c683:        done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:37159c5154b88277f12fe9aa20d728ca5c92fd38e6e707660ee27eef281de923:        done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:b788fa0813576d69f1efd12e893b97db7d007b5b93ca1cce6663f96ba1ab6488:        done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:4fe2ade4980c2dda4fc95858ebb981489baec8c1e4bd282ab1c3560be8ff9bde:        done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:798a8ef97f6bac88a42e34bf15d8e412242025372c4f567590579ba37d381b2c:        downloading    |#033[32m+++++++++++++++++++++#033[0m-----------------|  7.9 MiB/13.7 MiB
Oct 26 13:59:17 ctl01 ctr[6590]: elapsed: 1.6 s                                                                        total:  13.8 M (8.6 MiB/s)
Oct 26 13:59:17 ctl01 ctr[6590]: docker-prod-local.artifactory.mirantis.com/mirantis/projectcalico/calico/node:v3.3.2: resolved       |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: manifest-sha256:4b3e3750deeb97cf6f68e5d021f60891a0562f7412efdb545599e6ea505eaf18:     done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: config-sha256:4e9be81e3a5948d40df6358fdae2cc0dde85a0085723666c50ee7d15427a9b48:       done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:3f90cdf570685ae358ac0456a9d07fffc96858427c71d754f82e222d96f1c683:        done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:37159c5154b88277f12fe9aa20d728ca5c92fd38e6e707660ee27eef281de923:        done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:b788fa0813576d69f1efd12e893b97db7d007b5b93ca1cce6663f96ba1ab6488:        done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:4fe2ade4980c2dda4fc95858ebb981489baec8c1e4bd282ab1c3560be8ff9bde:        done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:798a8ef97f6bac88a42e34bf15d8e412242025372c4f567590579ba37d381b2c:        downloading    |#033[32m++++++++++++++++++++++++++++#033[0m----------| 10.5 MiB/13.7 MiB
Oct 26 13:59:17 ctl01 ctr[6590]: elapsed: 1.7 s                                                                        total:  16.3 M (9.6 MiB/s)
Oct 26 13:59:17 ctl01 ctr[6590]: docker-prod-local.artifactory.mirantis.com/mirantis/projectcalico/calico/node:v3.3.2: resolved       |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: manifest-sha256:4b3e3750deeb97cf6f68e5d021f60891a0562f7412efdb545599e6ea505eaf18:     done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: config-sha256:4e9be81e3a5948d40df6358fdae2cc0dde85a0085723666c50ee7d15427a9b48:       done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:3f90cdf570685ae358ac0456a9d07fffc96858427c71d754f82e222d96f1c683:        done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:37159c5154b88277f12fe9aa20d728ca5c92fd38e6e707660ee27eef281de923:        done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:b788fa0813576d69f1efd12e893b97db7d007b5b93ca1cce6663f96ba1ab6488:        done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:4fe2ade4980c2dda4fc95858ebb981489baec8c1e4bd282ab1c3560be8ff9bde:        done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:798a8ef97f6bac88a42e34bf15d8e412242025372c4f567590579ba37d381b2c:        downloading    |#033[32m+++++++++++++++++++++++++++++++++++++#033[0m-| 13.6 MiB/13.7 MiB
Oct 26 13:59:17 ctl01 ctr[6590]: elapsed: 1.8 s                                                                        total:  19.5 M (10.8 MiB/s)
Oct 26 13:59:17 ctl01 ctr[6590]: docker-prod-local.artifactory.mirantis.com/mirantis/projectcalico/calico/node:v3.3.2: resolved       |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: manifest-sha256:4b3e3750deeb97cf6f68e5d021f60891a0562f7412efdb545599e6ea505eaf18:     done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: config-sha256:4e9be81e3a5948d40df6358fdae2cc0dde85a0085723666c50ee7d15427a9b48:       done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:3f90cdf570685ae358ac0456a9d07fffc96858427c71d754f82e222d96f1c683:        done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:37159c5154b88277f12fe9aa20d728ca5c92fd38e6e707660ee27eef281de923:        done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:b788fa0813576d69f1efd12e893b97db7d007b5b93ca1cce6663f96ba1ab6488:        done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:4fe2ade4980c2dda4fc95858ebb981489baec8c1e4bd282ab1c3560be8ff9bde:        done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:798a8ef97f6bac88a42e34bf15d8e412242025372c4f567590579ba37d381b2c:        downloading    |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m| 13.7 MiB/13.7 MiB
Oct 26 13:59:17 ctl01 ctr[6590]: elapsed: 1.9 s                                                                        total:  19.6 M (10.3 MiB/s)
Oct 26 13:59:17 ctl01 ctr[6590]: docker-prod-local.artifactory.mirantis.com/mirantis/projectcalico/calico/node:v3.3.2: resolved       |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: manifest-sha256:4b3e3750deeb97cf6f68e5d021f60891a0562f7412efdb545599e6ea505eaf18:     done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: config-sha256:4e9be81e3a5948d40df6358fdae2cc0dde85a0085723666c50ee7d15427a9b48:       done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:3f90cdf570685ae358ac0456a9d07fffc96858427c71d754f82e222d96f1c683:        done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:37159c5154b88277f12fe9aa20d728ca5c92fd38e6e707660ee27eef281de923:        done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:b788fa0813576d69f1efd12e893b97db7d007b5b93ca1cce6663f96ba1ab6488:        done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:4fe2ade4980c2dda4fc95858ebb981489baec8c1e4bd282ab1c3560be8ff9bde:        done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: layer-sha256:798a8ef97f6bac88a42e34bf15d8e412242025372c4f567590579ba37d381b2c:        done           |#033[32m++++++++++++++++++++++++++++++++++++++#033[0m|
Oct 26 13:59:17 ctl01 ctr[6590]: elapsed: 2.0 s                                                                        total:  19.6 M (9.8 MiB/s)
Oct 26 13:59:17 ctl01 ctr[6590]: unpacking linux/amd64 sha256:4b3e3750deeb97cf6f68e5d021f60891a0562f7412efdb545599e6ea505eaf18...
Oct 26 13:59:19 ctl01 etcd[5203]: proto: no coders for int
Oct 26 13:59:19 ctl01 etcd[5203]: proto: no encoder for ValueSize int [GetProperties]
Oct 26 13:59:19 ctl01 ctr[6590]: done
Oct 26 13:59:19 ctl01 systemd[1]: Started calico-node.
Oct 26 13:59:19 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-active', 'calico-node.service'] in directory '/root'
Oct 26 13:59:19 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-enabled', 'calico-node.service'] in directory '/root'
Oct 26 13:59:19 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-enabled', 'calico-node.service'] in directory '/root'
Oct 26 13:59:20 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemd-run', '--scope', 'systemctl', 'enable', 'calico-node.service'] in directory '/root'
Oct 26 13:59:20 ctl01 systemd[1]: Started /bin/systemctl enable calico-node.service.
Oct 26 13:59:20 ctl01 systemd[1]: Reloading.
Oct 26 13:59:20 ctl01 containerd[6508]: time="2019-10-26T13:59:20.042123373Z" level=info msg="shim containerd-shim started" address="/containerd-shim/default/calico-node/shim.sock" debug=false pid=6664
Oct 26 13:59:20 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-enabled', 'calico-node.service'] in directory '/root'
Oct 26 13:59:20 ctl01 systemd[1]: Started ACPI event daemon.
Oct 26 13:59:20 ctl01 salt-minion[4526]: [INFO    ] {'calico-node': True}
Oct 26 13:59:20 ctl01 salt-minion[4526]: [INFO    ] Completed state [calico-node] at time 13:59:20.169390 duration_in_ms=4653.158
Oct 26 13:59:20 ctl01 salt-minion[4526]: [INFO    ] Running state [curl] at time 13:59:20.171315
Oct 26 13:59:20 ctl01 salt-minion[4526]: [INFO    ] Executing state pkg.installed for [curl]
Oct 26 13:59:20 ctl01 ctr[6630]: 2019-10-26 13:59:20.182 [INFO][8] startup.go 264: Early log level set to info
Oct 26 13:59:20 ctl01 ctr[6630]: 2019-10-26 13:59:20.182 [INFO][8] startup.go 280: Using NODENAME environment for node name
Oct 26 13:59:20 ctl01 ctr[6630]: 2019-10-26 13:59:20.182 [INFO][8] startup.go 292: Determined node name: ctl01
Oct 26 13:59:20 ctl01 ctr[6630]: 2019-10-26 13:59:20.230 [INFO][8] startup.go 105: Skipping datastore connection test
Oct 26 13:59:20 ctl01 ctr[6630]: 2019-10-26 13:59:20.231 [INFO][8] startup.go 365: Building new node resource Name="ctl01"
Oct 26 13:59:20 ctl01 ctr[6630]: 2019-10-26 13:59:20.231 [INFO][8] startup.go 380: Initialize BGP data
Oct 26 13:59:20 ctl01 ctr[6630]: 2019-10-26 13:59:20.232 [INFO][8] startup.go 474: Using IPv4 address from environment: IP=172.16.10.36
Oct 26 13:59:20 ctl01 ctr[6630]: 2019-10-26 13:59:20.233 [INFO][8] startup.go 507: IPv4 address 172.16.10.36 discovered on interface br-mgmt
Oct 26 13:59:20 ctl01 ctr[6630]: 2019-10-26 13:59:20.233 [INFO][8] startup.go 450: Node IPv4 changed, will check for conflicts
Oct 26 13:59:20 ctl01 ctr[6630]: 2019-10-26 13:59:20.234 [INFO][8] startup.go 640: Using AS number specified in environment (AS=64512)
Oct 26 13:59:20 ctl01 ctr[6630]: 2019-10-26 13:59:20.244 [INFO][8] startup.go 189: Using node name: ctl01
Oct 26 13:59:20 ctl01 salt-minion[4526]: [INFO    ] All specified packages are already installed
Oct 26 13:59:20 ctl01 salt-minion[4526]: [INFO    ] Completed state [curl] at time 13:59:20.368636 duration_in_ms=197.321
Oct 26 13:59:20 ctl01 salt-minion[4526]: [INFO    ] Running state [git] at time 13:59:20.368962
Oct 26 13:59:20 ctl01 salt-minion[4526]: [INFO    ] Executing state pkg.installed for [git]
Oct 26 13:59:20 ctl01 ctr[6630]: Calico node started successfully
Oct 26 13:59:20 ctl01 salt-minion[4526]: [INFO    ] All specified packages are already installed
Oct 26 13:59:20 ctl01 salt-minion[4526]: [INFO    ] Completed state [git] at time 13:59:20.382917 duration_in_ms=13.954
Oct 26 13:59:20 ctl01 salt-minion[4526]: [INFO    ] Running state [apt-transport-https] at time 13:59:20.383180
Oct 26 13:59:20 ctl01 salt-minion[4526]: [INFO    ] Executing state pkg.installed for [apt-transport-https]
Oct 26 13:59:20 ctl01 salt-minion[4526]: [INFO    ] All specified packages are already installed
Oct 26 13:59:20 ctl01 salt-minion[4526]: [INFO    ] Completed state [apt-transport-https] at time 13:59:20.392576 duration_in_ms=9.396
Oct 26 13:59:20 ctl01 salt-minion[4526]: [INFO    ] Running state [python-apt] at time 13:59:20.392818
Oct 26 13:59:20 ctl01 salt-minion[4526]: [INFO    ] Executing state pkg.installed for [python-apt]
Oct 26 13:59:20 ctl01 salt-minion[4526]: [INFO    ] All specified packages are already installed
Oct 26 13:59:20 ctl01 salt-minion[4526]: [INFO    ] Completed state [python-apt] at time 13:59:20.402175 duration_in_ms=9.357
Oct 26 13:59:20 ctl01 salt-minion[4526]: [INFO    ] Running state [socat] at time 13:59:20.402422
Oct 26 13:59:20 ctl01 salt-minion[4526]: [INFO    ] Executing state pkg.installed for [socat]
Oct 26 13:59:20 ctl01 salt-minion[4526]: [INFO    ] Executing command ['dpkg', '--get-selections', '*'] in directory '/root'
Oct 26 13:59:20 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemd-run', '--scope', 'apt-get', '-q', '-y', '-o', 'DPkg::Options::=--force-confold', '-o', 'DPkg::Options::=--force-confdef', 'install', 'socat'] in directory '/root'
Oct 26 13:59:20 ctl01 systemd[1]: Started /usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold -o DPkg::Options::=--force-confdef install socat.
Oct 26 13:59:21 ctl01 kernel: [  159.292978] Netfilter messages via NETLINK v0.30.
Oct 26 13:59:21 ctl01 kernel: [  159.297231] ip_set: protocol 6
Oct 26 13:59:21 ctl01 kernel: [  159.492415] ip6_tables: (C) 2000-2006 Netfilter Core Team
Oct 26 13:59:21 ctl01 kernel: [  159.493660] ip_tables: (C) 2000-2006 Netfilter Core Team
Oct 26 13:59:22 ctl01 salt-minion[4526]: [INFO    ] Executing command ['dpkg-query', '--showformat', '${Status} ${Package} ${Version} ${Architecture}', '-W'] in directory '/root'
Oct 26 13:59:22 ctl01 salt-minion[4526]: [INFO    ] Made the following changes:
Oct 26 13:59:22 ctl01 salt-minion[4526]: 'socat' changed from 'absent' to '1.7.3.1-1'
Oct 26 13:59:22 ctl01 salt-minion[4526]: [INFO    ] Loading fresh modules for state activity
Oct 26 13:59:22 ctl01 salt-minion[4526]: [INFO    ] Completed state [socat] at time 13:59:22.922471 duration_in_ms=2520.047
Oct 26 13:59:22 ctl01 salt-minion[4526]: [INFO    ] Running state [openssl] at time 13:59:22.928467
Oct 26 13:59:22 ctl01 salt-minion[4526]: [INFO    ] Executing state pkg.installed for [openssl]
Oct 26 13:59:23 ctl01 salt-minion[4526]: [INFO    ] All specified packages are already installed
Oct 26 13:59:23 ctl01 salt-minion[4526]: [INFO    ] Completed state [openssl] at time 13:59:23.650937 duration_in_ms=722.469
Oct 26 13:59:23 ctl01 salt-minion[4526]: [INFO    ] Running state [conntrack] at time 13:59:23.651233
Oct 26 13:59:23 ctl01 salt-minion[4526]: [INFO    ] Executing state pkg.installed for [conntrack]
Oct 26 13:59:23 ctl01 salt-minion[4526]: [INFO    ] Executing command ['dpkg', '--get-selections', '*'] in directory '/root'
Oct 26 13:59:23 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemd-run', '--scope', 'apt-get', '-q', '-y', '-o', 'DPkg::Options::=--force-confold', '-o', 'DPkg::Options::=--force-confdef', 'install', 'conntrack'] in directory '/root'
Oct 26 13:59:23 ctl01 systemd[1]: Started /usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold -o DPkg::Options::=--force-confdef install conntrack.
Oct 26 13:59:25 ctl01 salt-minion[4526]: [INFO    ] Executing command ['dpkg-query', '--showformat', '${Status} ${Package} ${Version} ${Architecture}', '-W'] in directory '/root'
Oct 26 13:59:25 ctl01 salt-minion[4526]: [INFO    ] Made the following changes:
Oct 26 13:59:25 ctl01 salt-minion[4526]: 'conntrack' changed from 'absent' to '1:1.4.3-3'
Oct 26 13:59:25 ctl01 salt-minion[4526]: [INFO    ] Loading fresh modules for state activity
Oct 26 13:59:26 ctl01 salt-minion[4526]: [INFO    ] Completed state [conntrack] at time 13:59:26.032697 duration_in_ms=2381.463
Oct 26 13:59:26 ctl01 salt-minion[4526]: [INFO    ] Running state [/usr/bin/hyperkube] at time 13:59:26.036320
Oct 26 13:59:26 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/usr/bin/hyperkube]
Oct 26 13:59:36 ctl01 salt-minion[4526]: [INFO    ] User sudo_ubuntu Executing command saltutil.find_job with jid 20191026135936602751
Oct 26 13:59:36 ctl01 salt-minion[4526]: [INFO    ] Starting a new job with PID 7723
Oct 26 13:59:36 ctl01 salt-minion[4526]: [INFO    ] Returning information for job: 20191026135936602751
Oct 26 13:59:37 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:59:37 ctl01 salt-minion[4526]: New file
Oct 26 13:59:37 ctl01 salt-minion[4526]: [INFO    ] Completed state [/usr/bin/hyperkube] at time 13:59:37.058790 duration_in_ms=11022.467
Oct 26 13:59:37 ctl01 salt-minion[4526]: [INFO    ] Running state [/usr/bin/kubectl] at time 13:59:37.060389
Oct 26 13:59:37 ctl01 salt-minion[4526]: [INFO    ] Executing state file.symlink for [/usr/bin/kubectl]
Oct 26 13:59:37 ctl01 salt-minion[4526]: [INFO    ] {'new': '/usr/bin/kubectl'}
Oct 26 13:59:37 ctl01 salt-minion[4526]: [INFO    ] Loading fresh modules for state activity
Oct 26 13:59:37 ctl01 salt-minion[4526]: [INFO    ] Completed state [/usr/bin/kubectl] at time 13:59:37.153089 duration_in_ms=92.7
Oct 26 13:59:37 ctl01 salt-minion[4526]: [INFO    ] Running state [/tmp/crictl] at time 13:59:37.157569
Oct 26 13:59:37 ctl01 salt-minion[4526]: [INFO    ] Executing state archive.extracted for [/tmp/crictl]
Oct 26 13:59:39 ctl01 salt-minion[4526]: [INFO    ] Executing command ['tar', 'xz', '-f', '/var/cache/salt/minion/extrn_files/base/github.com/kubernetes-sigs/cri-tools/releases/download/v1.12.0/crictl-v1.12.0-linux-amd64.tar.gz'] in directory '/tmp/crictl/'
Oct 26 13:59:39 ctl01 salt-minion[4526]: [INFO    ] Executing command ['tar', '--version'] in directory '/root'
Oct 26 13:59:39 ctl01 salt-minion[4526]: [INFO    ] {'extracted_files': 'no tar output so far', 'directories_created': ['/tmp/crictl/']}
Oct 26 13:59:39 ctl01 salt-minion[4526]: [INFO    ] Completed state [/tmp/crictl] at time 13:59:39.624177 duration_in_ms=2466.604
Oct 26 13:59:39 ctl01 salt-minion[4526]: [INFO    ] Running state [/usr/local/bin/crictl] at time 13:59:39.625688
Oct 26 13:59:39 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/usr/local/bin/crictl]
Oct 26 13:59:39 ctl01 salt-minion[4526]: [WARNING ] Use of argument owner found, "owner" is invalid, please use "user"
Oct 26 13:59:39 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:59:39 ctl01 salt-minion[4526]: New file
Oct 26 13:59:39 ctl01 salt-minion[4526]: [INFO    ] Completed state [/usr/local/bin/crictl] at time 13:59:39.960655 duration_in_ms=334.966
Oct 26 13:59:39 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/crictl.yaml] at time 13:59:39.960951
Oct 26 13:59:39 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/crictl.yaml]
Oct 26 13:59:39 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:59:39 ctl01 salt-minion[4526]: New file
Oct 26 13:59:39 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/crictl.yaml] at time 13:59:39.964770 duration_in_ms=3.817
Oct 26 13:59:39 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/criproxy] at time 13:59:39.965171
Oct 26 13:59:39 ctl01 salt-minion[4526]: [INFO    ] Executing state file.absent for [/etc/criproxy]
Oct 26 13:59:39 ctl01 salt-minion[4526]: [INFO    ] File /etc/criproxy is not present
Oct 26 13:59:39 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/criproxy] at time 13:59:39.966449 duration_in_ms=1.278
Oct 26 13:59:40 ctl01 salt-minion[4526]: [INFO    ] Running state [criproxy] at time 13:59:40.466305
Oct 26 13:59:40 ctl01 salt-minion[4526]: [INFO    ] Executing state service.dead for [criproxy]
Oct 26 13:59:40 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'status', 'criproxy.service', '-n', '0'] in directory '/root'
Oct 26 13:59:40 ctl01 salt-minion[4526]: [INFO    ] The named service criproxy is not available
Oct 26 13:59:40 ctl01 salt-minion[4526]: [INFO    ] Completed state [criproxy] at time 13:59:40.487330 duration_in_ms=21.025
Oct 26 13:59:40 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/systemd/system/kubelet.service] at time 13:59:40.487734
Oct 26 13:59:40 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/systemd/system/kubelet.service]
Oct 26 13:59:40 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/systemd/kubelet.service'
Oct 26 13:59:40 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:59:40 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:59:40 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:59:40 ctl01 salt-minion[4526]: New file
Oct 26 13:59:40 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/systemd/system/kubelet.service] at time 13:59:40.850125 duration_in_ms=362.389
Oct 26 13:59:40 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/config] at time 13:59:40.850615
Oct 26 13:59:40 ctl01 salt-minion[4526]: [INFO    ] Executing state file.absent for [/etc/kubernetes/config]
Oct 26 13:59:40 ctl01 salt-minion[4526]: [INFO    ] File /etc/kubernetes/config is not present
Oct 26 13:59:40 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/config] at time 13:59:40.852126 duration_in_ms=1.511
Oct 26 13:59:40 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/default/kubelet] at time 13:59:40.852503
Oct 26 13:59:40 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/default/kubelet]
Oct 26 13:59:40 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/kubelet/default.master'
Oct 26 13:59:40 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:59:41 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:59:41 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:59:41 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:59:41 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:59:41 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:59:41 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:59:41 ctl01 salt-minion[4526]: New file
Oct 26 13:59:41 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/default/kubelet] at time 13:59:41.751686 duration_in_ms=899.181
Oct 26 13:59:41 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/kubelet.kubeconfig] at time 13:59:41.752164
Oct 26 13:59:41 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/kubernetes/kubelet.kubeconfig]
Oct 26 13:59:41 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/kubelet/kubelet.kubeconfig.master'
Oct 26 13:59:41 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:59:41 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:59:42 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:59:42 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:59:42 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:59:42 ctl01 salt-minion[4526]: New file
Oct 26 13:59:42 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/kubelet.kubeconfig] at time 13:59:42.364993 duration_in_ms=612.81
Oct 26 13:59:42 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/manifests] at time 13:59:42.365468
Oct 26 13:59:42 ctl01 salt-minion[4526]: [INFO    ] Executing state file.directory for [/etc/kubernetes/manifests]
Oct 26 13:59:42 ctl01 salt-minion[4526]: [INFO    ] {'/etc/kubernetes/manifests': 'New Dir'}
Oct 26 13:59:42 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/manifests] at time 13:59:42.368167 duration_in_ms=2.699
Oct 26 13:59:42 ctl01 salt-minion[4526]: [INFO    ] Running state [kubelet] at time 13:59:42.370615
Oct 26 13:59:42 ctl01 salt-minion[4526]: [INFO    ] Executing state service.running for [kubelet]
Oct 26 13:59:42 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'status', 'kubelet.service', '-n', '0'] in directory '/root'
Oct 26 13:59:42 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-active', 'kubelet.service'] in directory '/root'
Oct 26 13:59:42 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-enabled', 'kubelet.service'] in directory '/root'
Oct 26 13:59:42 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemd-run', '--scope', 'systemctl', 'start', 'kubelet.service'] in directory '/root'
Oct 26 13:59:42 ctl01 systemd[1]: Started /bin/systemctl start kubelet.service.
Oct 26 13:59:42 ctl01 systemd[1]: Started Kubernetes Kubelet Server.
Oct 26 13:59:42 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-active', 'kubelet.service'] in directory '/root'
Oct 26 13:59:42 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-enabled', 'kubelet.service'] in directory '/root'
Oct 26 13:59:42 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-enabled', 'kubelet.service'] in directory '/root'
Oct 26 13:59:42 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemd-run', '--scope', 'systemctl', 'enable', 'kubelet.service'] in directory '/root'
Oct 26 13:59:42 ctl01 systemd[1]: Started /bin/systemctl enable kubelet.service.
Oct 26 13:59:42 ctl01 systemd[1]: Reloading.
Oct 26 13:59:42 ctl01 kubelet[8020]: Flag --pod-manifest-path has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 26 13:59:42 ctl01 kubelet[8020]: Flag --address has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 26 13:59:42 ctl01 kubelet[8020]: Flag --allow-privileged has been deprecated, will be removed in a future version
Oct 26 13:59:42 ctl01 kubelet[8020]: Flag --cluster-dns has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 26 13:59:42 ctl01 kubelet[8020]: Flag --cluster-domain has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 26 13:59:42 ctl01 kubelet[8020]: Flag --fail-swap-on has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579649    8020 flags.go:33] FLAG: --address="172.16.10.36"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579700    8020 flags.go:33] FLAG: --allow-privileged="true"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579709    8020 flags.go:33] FLAG: --allowed-unsafe-sysctls="[]"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579718    8020 flags.go:33] FLAG: --alsologtostderr="false"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579726    8020 flags.go:33] FLAG: --anonymous-auth="true"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579732    8020 flags.go:33] FLAG: --application-metrics-count-limit="100"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579739    8020 flags.go:33] FLAG: --authentication-token-webhook="false"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579744    8020 flags.go:33] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579752    8020 flags.go:33] FLAG: --authorization-mode="AlwaysAllow"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579759    8020 flags.go:33] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579765    8020 flags.go:33] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579770    8020 flags.go:33] FLAG: --azure-container-registry-config=""
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579776    8020 flags.go:33] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579783    8020 flags.go:33] FLAG: --bootstrap-checkpoint-path=""
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579788    8020 flags.go:33] FLAG: --bootstrap-kubeconfig=""
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579793    8020 flags.go:33] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579799    8020 flags.go:33] FLAG: --cgroup-driver="cgroupfs"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579805    8020 flags.go:33] FLAG: --cgroup-root=""
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579810    8020 flags.go:33] FLAG: --cgroups-per-qos="true"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579816    8020 flags.go:33] FLAG: --chaos-chance="0"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579824    8020 flags.go:33] FLAG: --client-ca-file=""
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579830    8020 flags.go:33] FLAG: --cloud-config=""
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579835    8020 flags.go:33] FLAG: --cloud-provider=""
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579840    8020 flags.go:33] FLAG: --cluster-dns="[10.254.0.10]"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579853    8020 flags.go:33] FLAG: --cluster-domain="cluster.local"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579859    8020 flags.go:33] FLAG: --cni-bin-dir="/opt/cni/bin"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579864    8020 flags.go:33] FLAG: --cni-conf-dir="/etc/cni/net.d"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579870    8020 flags.go:33] FLAG: --config=""
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579876    8020 flags.go:33] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579883    8020 flags.go:33] FLAG: --container-log-max-files="5"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579893    8020 flags.go:33] FLAG: --container-log-max-size="10Mi"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579899    8020 flags.go:33] FLAG: --container-runtime="remote"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579904    8020 flags.go:33] FLAG: --container-runtime-endpoint="unix:///run/containerd/containerd.sock"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579916    8020 flags.go:33] FLAG: --containerd="unix:///var/run/containerd.sock"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579923    8020 flags.go:33] FLAG: --containerized="false"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579928    8020 flags.go:33] FLAG: --contention-profiling="false"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579934    8020 flags.go:33] FLAG: --cpu-cfs-quota="true"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579939    8020 flags.go:33] FLAG: --cpu-cfs-quota-period="100ms"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579945    8020 flags.go:33] FLAG: --cpu-manager-policy="none"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579951    8020 flags.go:33] FLAG: --cpu-manager-reconcile-period="10s"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579957    8020 flags.go:33] FLAG: --docker="unix:///var/run/docker.sock"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579963    8020 flags.go:33] FLAG: --docker-endpoint="unix:///var/run/docker.sock"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579969    8020 flags.go:33] FLAG: --docker-env-metadata-whitelist=""
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579974    8020 flags.go:33] FLAG: --docker-only="false"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579980    8020 flags.go:33] FLAG: --docker-root="/var/lib/docker"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579986    8020 flags.go:33] FLAG: --docker-tls="false"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579992    8020 flags.go:33] FLAG: --docker-tls-ca="ca.pem"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.579997    8020 flags.go:33] FLAG: --docker-tls-cert="cert.pem"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580003    8020 flags.go:33] FLAG: --docker-tls-key="key.pem"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580008    8020 flags.go:33] FLAG: --dynamic-config-dir=""
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580016    8020 flags.go:33] FLAG: --enable-controller-attach-detach="true"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580021    8020 flags.go:33] FLAG: --enable-debugging-handlers="true"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580027    8020 flags.go:33] FLAG: --enable-load-reader="false"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580032    8020 flags.go:33] FLAG: --enable-server="true"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580038    8020 flags.go:33] FLAG: --enforce-node-allocatable="[pods]"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580045    8020 flags.go:33] FLAG: --event-burst="10"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580051    8020 flags.go:33] FLAG: --event-qps="5"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580057    8020 flags.go:33] FLAG: --event-storage-age-limit="default=0"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580062    8020 flags.go:33] FLAG: --event-storage-event-limit="default=0"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580068    8020 flags.go:33] FLAG: --eviction-hard="imagefs.available<15%,memory.available<100Mi,nodefs.available<10%,nodefs.inodesFree<5%"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580084    8020 flags.go:33] FLAG: --eviction-max-pod-grace-period="0"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580090    8020 flags.go:33] FLAG: --eviction-minimum-reclaim=""
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580104    8020 flags.go:33] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580110    8020 flags.go:33] FLAG: --eviction-soft=""
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580116    8020 flags.go:33] FLAG: --eviction-soft-grace-period=""
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580127    8020 flags.go:33] FLAG: --exit-on-lock-contention="false"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580133    8020 flags.go:33] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580138    8020 flags.go:33] FLAG: --experimental-bootstrap-kubeconfig=""
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580144    8020 flags.go:33] FLAG: --experimental-check-node-capabilities-before-mount="false"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580150    8020 flags.go:33] FLAG: --experimental-dockershim="false"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580155    8020 flags.go:33] FLAG: --experimental-dockershim-root-directory="/var/lib/dockershim"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580161    8020 flags.go:33] FLAG: --experimental-fail-swap-on="true"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580167    8020 flags.go:33] FLAG: --experimental-kernel-memcg-notification="false"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580172    8020 flags.go:33] FLAG: --experimental-mounter-path=""
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580177    8020 flags.go:33] FLAG: --fail-swap-on="true"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580183    8020 flags.go:33] FLAG: --feature-gates=""
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580191    8020 flags.go:33] FLAG: --file-check-frequency="20s"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580197    8020 flags.go:33] FLAG: --global-housekeeping-interval="1m0s"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580203    8020 flags.go:33] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580209    8020 flags.go:33] FLAG: --healthz-bind-address="127.0.0.1"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580215    8020 flags.go:33] FLAG: --healthz-port="10248"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580221    8020 flags.go:33] FLAG: --help="false"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580226    8020 flags.go:33] FLAG: --host-ipc-sources="[*]"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580236    8020 flags.go:33] FLAG: --host-network-sources="[*]"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580243    8020 flags.go:33] FLAG: --host-pid-sources="[*]"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580252    8020 flags.go:33] FLAG: --hostname-override="ctl01"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580258    8020 flags.go:33] FLAG: --housekeeping-interval="10s"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580264    8020 flags.go:33] FLAG: --http-check-frequency="20s"
Oct 26 13:59:42 ctl01 systemd[1]: kubelet.service: Dependency Conflicts=cadvisor.service dropped, merged into kubelet.service
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580270    8020 flags.go:33] FLAG: --image-gc-high-threshold="85"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580276    8020 flags.go:33] FLAG: --image-gc-low-threshold="80"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580281    8020 flags.go:33] FLAG: --image-pull-progress-deadline="1m0s"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580287    8020 flags.go:33] FLAG: --image-service-endpoint=""
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580292    8020 flags.go:33] FLAG: --iptables-drop-bit="15"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580298    8020 flags.go:33] FLAG: --iptables-masquerade-bit="14"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580309    8020 flags.go:33] FLAG: --keep-terminated-pod-volumes="false"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580315    8020 flags.go:33] FLAG: --kube-api-burst="10"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580320    8020 flags.go:33] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580333    8020 flags.go:33] FLAG: --kube-api-qps="5"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580338    8020 flags.go:33] FLAG: --kube-reserved=""
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580345    8020 flags.go:33] FLAG: --kube-reserved-cgroup=""
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580350    8020 flags.go:33] FLAG: --kubeconfig="/etc/kubernetes/kubelet.kubeconfig"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580356    8020 flags.go:33] FLAG: --kubelet-cgroups=""
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580362    8020 flags.go:33] FLAG: --lock-file=""
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580367    8020 flags.go:33] FLAG: --log-backtrace-at=":0"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580374    8020 flags.go:33] FLAG: --log-cadvisor-usage="false"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580380    8020 flags.go:33] FLAG: --log-dir=""
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580385    8020 flags.go:33] FLAG: --log-file=""
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580391    8020 flags.go:33] FLAG: --log-flush-frequency="5s"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580397    8020 flags.go:33] FLAG: --logtostderr="true"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580402    8020 flags.go:33] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580409    8020 flags.go:33] FLAG: --make-iptables-util-chains="true"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580415    8020 flags.go:33] FLAG: --manifest-url=""
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580420    8020 flags.go:33] FLAG: --manifest-url-header=""
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580429    8020 flags.go:33] FLAG: --master-service-namespace="default"
Oct 26 13:59:42 ctl01 systemd[1]: kubelet.service: Dependency ConflictedBy=cadvisor.service dropped, merged into kubelet.service
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580435    8020 flags.go:33] FLAG: --max-open-files="1000000"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580443    8020 flags.go:33] FLAG: --max-pods="110"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580449    8020 flags.go:33] FLAG: --maximum-dead-containers="-1"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580455    8020 flags.go:33] FLAG: --maximum-dead-containers-per-container="1"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580460    8020 flags.go:33] FLAG: --minimum-container-ttl-duration="0s"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580466    8020 flags.go:33] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580472    8020 flags.go:33] FLAG: --network-plugin="cni"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580478    8020 flags.go:33] FLAG: --network-plugin-mtu="0"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580483    8020 flags.go:33] FLAG: --node-ip="172.16.10.36"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580489    8020 flags.go:33] FLAG: --node-labels="node-role.kubernetes.io/master=true"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580500    8020 flags.go:33] FLAG: --node-status-max-images="50"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580505    8020 flags.go:33] FLAG: --node-status-update-frequency="10s"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580518    8020 flags.go:33] FLAG: --non-masquerade-cidr="10.0.0.0/8"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580524    8020 flags.go:33] FLAG: --oom-score-adj="-999"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580530    8020 flags.go:33] FLAG: --pod-cidr=""
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580541    8020 flags.go:33] FLAG: --pod-infra-container-image="docker-prod-local.artifactory.mirantis.com/mirantis/kubernetes/pause-amd64:v1.13.5-3"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580550    8020 flags.go:33] FLAG: --pod-manifest-path="/etc/kubernetes/manifests"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580556    8020 flags.go:33] FLAG: --pod-max-pids="-1"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580562    8020 flags.go:33] FLAG: --pods-per-core="0"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580568    8020 flags.go:33] FLAG: --port="10250"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580573    8020 flags.go:33] FLAG: --protect-kernel-defaults="false"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580579    8020 flags.go:33] FLAG: --provider-id=""
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580584    8020 flags.go:33] FLAG: --qos-reserved=""
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580590    8020 flags.go:33] FLAG: --read-only-port="10255"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580596    8020 flags.go:33] FLAG: --really-crash-for-testing="false"
Oct 26 13:59:42 ctl01 systemd[1]: Started ACPI event daemon.
Oct 26 13:59:42 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-enabled', 'kubelet.service'] in directory '/root'
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580602    8020 flags.go:33] FLAG: --redirect-container-streaming="false"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580698    8020 flags.go:33] FLAG: --register-node="true"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580705    8020 flags.go:33] FLAG: --register-schedulable="true"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580710    8020 flags.go:33] FLAG: --register-with-taints=""
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580718    8020 flags.go:33] FLAG: --registry-burst="10"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580724    8020 flags.go:33] FLAG: --registry-qps="5"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580730    8020 flags.go:33] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580736    8020 flags.go:33] FLAG: --root-dir="/var/lib/kubelet"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580742    8020 flags.go:33] FLAG: --rotate-certificates="false"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580747    8020 flags.go:33] FLAG: --rotate-server-certificates="false"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580753    8020 flags.go:33] FLAG: --runonce="false"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580758    8020 flags.go:33] FLAG: --runtime-cgroups=""
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580764    8020 flags.go:33] FLAG: --runtime-request-timeout="2m0s"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580770    8020 flags.go:33] FLAG: --seccomp-profile-root="/var/lib/kubelet/seccomp"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580776    8020 flags.go:33] FLAG: --serialize-image-pulls="true"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580782    8020 flags.go:33] FLAG: --stderrthreshold="2"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580787    8020 flags.go:33] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580793    8020 flags.go:33] FLAG: --storage-driver-db="cadvisor"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580799    8020 flags.go:33] FLAG: --storage-driver-host="localhost:8086"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580812    8020 flags.go:33] FLAG: --storage-driver-password="root"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580818    8020 flags.go:33] FLAG: --storage-driver-secure="false"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580824    8020 flags.go:33] FLAG: --storage-driver-table="stats"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580833    8020 flags.go:33] FLAG: --storage-driver-user="root"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580839    8020 flags.go:33] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580845    8020 flags.go:33] FLAG: --sync-frequency="1m0s"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580851    8020 flags.go:33] FLAG: --system-cgroups=""
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580856    8020 flags.go:33] FLAG: --system-reserved=""
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580863    8020 flags.go:33] FLAG: --system-reserved-cgroup=""
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580868    8020 flags.go:33] FLAG: --tls-cert-file=""
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580874    8020 flags.go:33] FLAG: --tls-cipher-suites="[]"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580882    8020 flags.go:33] FLAG: --tls-min-version=""
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580888    8020 flags.go:33] FLAG: --tls-private-key-file=""
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580893    8020 flags.go:33] FLAG: --v="2"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580899    8020 flags.go:33] FLAG: --version="false"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580908    8020 flags.go:33] FLAG: --vmodule=""
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580914    8020 flags.go:33] FLAG: --volume-plugin-dir="/usr/libexec/kubernetes/kubelet-plugins/volume/exec/"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580920    8020 flags.go:33] FLAG: --volume-stats-agg-period="1m0s"
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.580960    8020 feature_gate.go:206] feature gates: &{map[]}
Oct 26 13:59:42 ctl01 kubelet[8020]: W1026 13:59:42.580993    8020 options.go:265] unknown 'kubernetes.io' or 'k8s.io' labels specified with --node-labels: [node-role.kubernetes.io/master]
Oct 26 13:59:42 ctl01 kubelet[8020]: W1026 13:59:42.581007    8020 options.go:266] in 1.15, --node-labels in the 'kubernetes.io' namespace must begin with an allowed prefix (kubelet.kubernetes.io, node.kubernetes.io) or be in the specifically allowed set (beta.kubernetes.io/arch, beta.kubernetes.io/instance-type, beta.kubernetes.io/os, failure-domain.beta.kubernetes.io/region, failure-domain.beta.kubernetes.io/zone, failure-domain.kubernetes.io/region, failure-domain.kubernetes.io/zone, kubernetes.io/arch, kubernetes.io/hostname, kubernetes.io/instance-type, kubernetes.io/os)
Oct 26 13:59:42 ctl01 kubelet[8020]: W1026 13:59:42.581018    8020 server.go:182] Warning: For remote container runtime, --pod-infra-container-image is ignored in kubelet, which should be set in that remote runtime instead
Oct 26 13:59:42 ctl01 kubelet[8020]: I1026 13:59:42.581075    8020 feature_gate.go:206] feature gates: &{map[]}
Oct 26 13:59:42 ctl01 salt-minion[4526]: [INFO    ] {'kubelet': True}
Oct 26 13:59:42 ctl01 salt-minion[4526]: [INFO    ] Completed state [kubelet] at time 13:59:42.628543 duration_in_ms=257.927
Oct 26 13:59:42 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/logrotate.d/kubernetes] at time 13:59:42.628909
Oct 26 13:59:42 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/logrotate.d/kubernetes]
Oct 26 13:59:42 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/logrotate'
Oct 26 13:59:42 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:59:42 ctl01 salt-minion[4526]: New file
Oct 26 13:59:42 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/logrotate.d/kubernetes] at time 13:59:42.656296 duration_in_ms=27.386
Oct 26 13:59:42 ctl01 salt-minion[4526]: [INFO    ] Running state [/opt/cni/bin] at time 13:59:42.656551
Oct 26 13:59:42 ctl01 salt-minion[4526]: [INFO    ] Executing state archive.extracted for [/opt/cni/bin]
Oct 26 13:59:43 ctl01 systemd[1]: Started Kubernetes systemd probe.
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.056414    8020 mount_linux.go:180] Detected OS with systemd
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.056483    8020 server.go:407] Version: v1.13.5-3+98374c02d2d8c1
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.056559    8020 feature_gate.go:206] feature gates: &{map[]}
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.056647    8020 feature_gate.go:206] feature gates: &{map[]}
Oct 26 13:59:43 ctl01 kubelet[8020]: W1026 13:59:43.056673    8020 options.go:265] unknown 'kubernetes.io' or 'k8s.io' labels specified with --node-labels: [node-role.kubernetes.io/master]
Oct 26 13:59:43 ctl01 kubelet[8020]: W1026 13:59:43.056692    8020 options.go:266] in 1.15, --node-labels in the 'kubernetes.io' namespace must begin with an allowed prefix (kubelet.kubernetes.io, node.kubernetes.io) or be in the specifically allowed set (beta.kubernetes.io/arch, beta.kubernetes.io/instance-type, beta.kubernetes.io/os, failure-domain.beta.kubernetes.io/region, failure-domain.beta.kubernetes.io/zone, failure-domain.kubernetes.io/region, failure-domain.kubernetes.io/zone, kubernetes.io/arch, kubernetes.io/hostname, kubernetes.io/instance-type, kubernetes.io/os)
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.056811    8020 plugins.go:103] No cloud provider specified.
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.056830    8020 server.go:523] No cloud provider specified: "" from the config file: ""
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.064748    8020 manager.go:155] cAdvisor running in container: "/sys/fs/cgroup/cpu,cpuacct/system.slice/kubelet.service"
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.066412    8020 fs.go:142] Filesystem UUIDs: map[2019-10-26-15-49-44-00:/dev/sr0 3dc7f99a-1e06-428f-86bf-f1f8932b4515:/dev/vda1]
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.066460    8020 fs.go:143] Filesystem partitions: map[tmpfs:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /dev/vda1:{mountpoint:/ major:252 minor:1 fsType:ext4 blockSize:0}]
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.078205    8020 manager.go:229] Machine: {NumCores:8 CpuFrequency:2799994 MemoryCapacity:14705016832 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:70a4fc47e8254bb0a9da2f2beeb1a9ad SystemUUID:B60BACF9-7C5C-402C-99A3-F64133E62AD6 BootID:f6b0e736-be69-43fe-8cfa-5f08c44a9aa3 Filesystems:[{Device:tmpfs DeviceMajor:0 DeviceMinor:24 Capacity:1470504960 Type:vfs Inodes:1795046 HasInodes:true} {Device:/dev/vda1 DeviceMajor:252 DeviceMinor:1 Capacity:104022159360 Type:vfs Inodes:12800000 HasInodes:true}] DiskMap:map[252:0:{Name:vda Major:252 Minor:0 Size:107374182400 Scheduler:none}] NetworkDevices:[{Name:br-mgmt MacAddress:52:54:00:cc:bf:d7 Speed:0 Mtu:9000} {Name:ens3 MacAddress:52:54:00:d1:44:f2 Speed:-1 Mtu:9000} {Name:ens4 MacAddress:52:54:00:cc:bf:d7 Speed:-1 Mtu:9000} {Name:ens5 MacAddress:52:54:00:dd:4a:72 Speed:-1 Mtu:9000} {Name:ens5.1000 MacAddress:52:54:00:dd:4a:72 Speed:-1 Mtu:9000} {Name:ens6 MacAddress:52:54:00:23:fb:5b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:14705016832 Cores:[{Id:0 Threads:[0] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]} {Id:1 Memory:0 Cores:[{Id:0 Threads:[1] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]} {Id:2 Memory:0 Cores:[{Id:0 Threads:[2] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]} {Id:3 Memory:0 Cores:[{Id:0 Threads:[3] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]} {Id:4 Memory:0 Cores:[{Id:0 Threads:[4] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]} {Id:5 Memory:0 Cores:[{Id:0 Threads:[5] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]} {Id:6 Memory:0 Cores:
Oct 26 13:59:43 ctl01 kubelet[8020]: [{Id:0 Threads:[6] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]} {Id:7 Memory:0 Cores:[{Id:0 Threads:[7] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.078612    8020 manager.go:235] Version: {KernelVersion:4.15.0-66-generic ContainerOsVersion:Ubuntu 16.04.6 LTS DockerVersion:Unknown DockerAPIVersion:Unknown CadvisorVersion: CadvisorRevision:}
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.078719    8020 server.go:666] --cgroups-per-qos enabled, but --cgroup-root was not specified.  defaulting to /
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.079003    8020 container_manager_linux.go:248] container manager verified user specified cgroup-root exists: []
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.079019    8020 container_manager_linux.go:253] Creating Container Manager object based on Node Config: {RuntimeCgroupsName: SystemCgroupsName: KubeletCgroupsName: ContainerRuntime:remote CgroupsPerQOS:true CgroupRoot:/ CgroupDriver:cgroupfs KubeletRootDir:/var/lib/kubelet ProtectKernelDefaults:false NodeAllocatableConfig:{KubeReservedCgroupName: SystemReservedCgroupName: EnforceNodeAllocatable:map[pods:{}] KubeReserved:map[] SystemReserved:map[] HardEvictionThresholds:[{Signal:memory.available Operator:LessThan Value:{Quantity:100Mi Percentage:0} GracePeriod:0s MinReclaim:<nil>} {Signal:nodefs.available Operator:LessThan Value:{Quantity:<nil> Percentage:0.1} GracePeriod:0s MinReclaim:<nil>} {Signal:nodefs.inodesFree Operator:LessThan Value:{Quantity:<nil> Percentage:0.05} GracePeriod:0s MinReclaim:<nil>} {Signal:imagefs.available Operator:LessThan Value:{Quantity:<nil> Percentage:0.15} GracePeriod:0s MinReclaim:<nil>}]} QOSReserved:map[] ExperimentalCPUManagerPolicy:none ExperimentalCPUManagerReconcilePeriod:10s ExperimentalPodPidsLimit:-1 EnforceCPULimits:true CPUCFSQuotaPeriod:100ms}
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.079171    8020 container_manager_linux.go:272] Creating device plugin manager: true
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.079183    8020 manager.go:109] Creating Device Plugin manager at /var/lib/kubelet/device-plugins/kubelet.sock
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.079288    8020 state_mem.go:36] [cpumanager] initializing new in-memory state store
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.082603    8020 server.go:941] Using root directory: /var/lib/kubelet
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.082644    8020 kubelet.go:281] Adding pod path: /etc/kubernetes/manifests
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.082673    8020 file.go:68] Watching path "/etc/kubernetes/manifests"
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.082708    8020 kubelet.go:306] Watching apiserver
Oct 26 13:59:43 ctl01 kubelet[8020]: E1026 13:59:43.083598    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://172.16.10.36:443/api/v1/pods?fieldSelector=spec.nodeName%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:43 ctl01 kubelet[8020]: E1026 13:59:43.083610    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://172.16.10.36:443/api/v1/nodes?fieldSelector=metadata.name%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:43 ctl01 kubelet[8020]: E1026 13:59:43.083976    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.091339    8020 kuberuntime_manager.go:198] Container runtime containerd initialized, version: 1.2.5-3, apiVersion: v1alpha2
Oct 26 13:59:43 ctl01 kubelet[8020]: W1026 13:59:43.091644    8020 probe.go:271] Flexvolume plugin directory at /usr/libexec/kubernetes/kubelet-plugins/volume/exec/ does not exist. Recreating.
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.091863    8020 plugins.go:547] Loaded volume plugin "kubernetes.io/aws-ebs"
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.091880    8020 plugins.go:547] Loaded volume plugin "kubernetes.io/empty-dir"
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.091890    8020 plugins.go:547] Loaded volume plugin "kubernetes.io/gce-pd"
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.091900    8020 plugins.go:547] Loaded volume plugin "kubernetes.io/git-repo"
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.091910    8020 plugins.go:547] Loaded volume plugin "kubernetes.io/host-path"
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.091919    8020 plugins.go:547] Loaded volume plugin "kubernetes.io/nfs"
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.091929    8020 plugins.go:547] Loaded volume plugin "kubernetes.io/secret"
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.091938    8020 plugins.go:547] Loaded volume plugin "kubernetes.io/iscsi"
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.091950    8020 plugins.go:547] Loaded volume plugin "kubernetes.io/glusterfs"
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.091967    8020 plugins.go:547] Loaded volume plugin "kubernetes.io/rbd"
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.091977    8020 plugins.go:547] Loaded volume plugin "kubernetes.io/cinder"
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.091986    8020 plugins.go:547] Loaded volume plugin "kubernetes.io/quobyte"
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.091995    8020 plugins.go:547] Loaded volume plugin "kubernetes.io/cephfs"
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.092007    8020 plugins.go:547] Loaded volume plugin "kubernetes.io/downward-api"
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.092016    8020 plugins.go:547] Loaded volume plugin "kubernetes.io/fc"
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.092026    8020 plugins.go:547] Loaded volume plugin "kubernetes.io/flocker"
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.092035    8020 plugins.go:547] Loaded volume plugin "kubernetes.io/azure-file"
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.092045    8020 plugins.go:547] Loaded volume plugin "kubernetes.io/configmap"
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.092055    8020 plugins.go:547] Loaded volume plugin "kubernetes.io/vsphere-volume"
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.092064    8020 plugins.go:547] Loaded volume plugin "kubernetes.io/azure-disk"
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.092074    8020 plugins.go:547] Loaded volume plugin "kubernetes.io/photon-pd"
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.092084    8020 plugins.go:547] Loaded volume plugin "kubernetes.io/projected"
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.092103    8020 plugins.go:547] Loaded volume plugin "kubernetes.io/portworx-volume"
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.092114    8020 plugins.go:547] Loaded volume plugin "kubernetes.io/scaleio"
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.092124    8020 plugins.go:547] Loaded volume plugin "kubernetes.io/local-volume"
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.092134    8020 plugins.go:547] Loaded volume plugin "kubernetes.io/storageos"
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.092148    8020 plugins.go:547] Loaded volume plugin "kubernetes.io/csi"
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.092403    8020 server.go:999] Started kubelet
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.092700    8020 server.go:137] Starting to listen on 172.16.10.36:10250
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.092767    8020 server.go:157] Starting to listen read-only on 172.16.10.36:10255
Oct 26 13:59:43 ctl01 kubelet[8020]: E1026 13:59:43.093074    8020 event.go:212] Unable to write event: 'Post https://172.16.10.36:443/api/v1/namespaces/default/events: dial tcp 172.16.10.36:443: connect: connection refused' (may retry after sleeping)
Oct 26 13:59:43 ctl01 kubelet[8020]: E1026 13:59:43.093201    8020 cri_stats_provider.go:320] Failed to get the info of the filesystem with mountpoint "/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs": unable to find data in memory cache.
Oct 26 13:59:43 ctl01 kubelet[8020]: E1026 13:59:43.093226    8020 kubelet.go:1308] Image garbage collection failed once. Stats initialization may not have completed yet: invalid capacity 0 on image filesystem
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.093307    8020 fs_resource_analyzer.go:66] Starting FS ResourceAnalyzer
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.093346    8020 status_manager.go:152] Starting to sync pod status with apiserver
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.093362    8020 kubelet.go:1829] Starting kubelet main sync loop.
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.093382    8020 kubelet.go:1846] skipping pod synchronization - [container runtime status check may not have completed yet PLEG is not healthy: pleg has yet to be successful]
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.093475    8020 volume_manager.go:246] The desired_state_of_world populator starts
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.093486    8020 volume_manager.go:248] Starting Kubelet Volume Manager
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.093509    8020 desired_state_of_world_populator.go:130] Desired state populator starts to run
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.094148    8020 server.go:333] Adding debug handlers to kubelet server.
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.095426    8020 factory.go:136] Registering containerd factory
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.097363    8020 factory.go:54] Registering systemd factory
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.097621    8020 factory.go:97] Registering Raw factory
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.097798    8020 manager.go:1222] Started watching for new ooms in manager
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.099090    8020 manager.go:365] Starting recovery of all containers
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.145757    8020 manager.go:370] Recovery completed
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.193547    8020 kubelet.go:1846] skipping pod synchronization - [container runtime status check may not have completed yet]
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.193593    8020 kubelet_node_status.go:279] Setting node annotation to enable volume controller attach/detach
Oct 26 13:59:43 ctl01 kubelet[8020]: E1026 13:59:43.193665    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.193898    8020 setters.go:72] Using node IP: "172.16.10.36"
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.194988    8020 kubelet_node_status.go:447] Recording NodeHasSufficientMemory event message for node ctl01
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.195050    8020 kubelet_node_status.go:447] Recording NodeHasNoDiskPressure event message for node ctl01
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.195078    8020 kubelet_node_status.go:447] Recording NodeHasSufficientPID event message for node ctl01
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.195114    8020 kubelet_node_status.go:72] Attempting to register node ctl01
Oct 26 13:59:43 ctl01 kubelet[8020]: E1026 13:59:43.195586    8020 kubelet_node_status.go:94] Unable to register node "ctl01" with API server: Post https://172.16.10.36:443/api/v1/nodes: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.206794    8020 kubelet_node_status.go:279] Setting node annotation to enable volume controller attach/detach
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.207030    8020 setters.go:72] Using node IP: "172.16.10.36"
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.207735    8020 kubelet_node_status.go:447] Recording NodeHasSufficientMemory event message for node ctl01
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.207780    8020 kubelet_node_status.go:447] Recording NodeHasNoDiskPressure event message for node ctl01
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.207794    8020 kubelet_node_status.go:447] Recording NodeHasSufficientPID event message for node ctl01
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.207825    8020 cpu_manager.go:155] [cpumanager] starting with none policy
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.207833    8020 cpu_manager.go:156] [cpumanager] reconciling every 10s
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.207852    8020 policy_none.go:42] [cpumanager] none policy: Start
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.208267    8020 container_manager_linux.go:376] Updating kernel flag: kernel/panic, expected value: 10, actual value: 60
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.208332    8020 container_manager_linux.go:376] Updating kernel flag: kernel/panic_on_oops, expected value: 1, actual value: 0
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.208440    8020 container_manager_linux.go:376] Updating kernel flag: vm/overcommit_memory, expected value: 1, actual value: 0
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.223718    8020 manager.go:196] Starting Device Plugin manager
Oct 26 13:59:43 ctl01 kubelet[8020]: W1026 13:59:43.223761    8020 manager.go:537] Failed to retrieve checkpoint for "kubelet_internal_checkpoint": checkpoint is not found
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.223891    8020 manager.go:231] Serving device plugin registration server on "/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.223945    8020 plugin_watcher.go:90] Plugin Watcher Start at /var/lib/kubelet/plugins_registry
Oct 26 13:59:43 ctl01 kubelet[8020]: E1026 13:59:43.224620    8020 eviction_manager.go:247] eviction manager: failed to get summary stats: failed to get node info: node "ctl01" not found
Oct 26 13:59:43 ctl01 kubelet[8020]: E1026 13:59:43.293801    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.393722    8020 kubelet.go:1908] SyncLoop (ADD, "file"): ""
Oct 26 13:59:43 ctl01 kubelet[8020]: E1026 13:59:43.394251    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.395759    8020 kubelet_node_status.go:279] Setting node annotation to enable volume controller attach/detach
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.396104    8020 setters.go:72] Using node IP: "172.16.10.36"
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.396882    8020 kubelet_node_status.go:447] Recording NodeHasSufficientMemory event message for node ctl01
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.396937    8020 kubelet_node_status.go:447] Recording NodeHasNoDiskPressure event message for node ctl01
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.396954    8020 kubelet_node_status.go:447] Recording NodeHasSufficientPID event message for node ctl01
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.396975    8020 kubelet_node_status.go:72] Attempting to register node ctl01
Oct 26 13:59:43 ctl01 kubelet[8020]: E1026 13:59:43.397484    8020 kubelet_node_status.go:94] Unable to register node "ctl01" with API server: Post https://172.16.10.36:443/api/v1/nodes: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:43 ctl01 kubelet[8020]: E1026 13:59:43.494405    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:43 ctl01 kubelet[8020]: E1026 13:59:43.594643    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:43 ctl01 kubelet[8020]: E1026 13:59:43.694901    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:43 ctl01 kubelet[8020]: E1026 13:59:43.795089    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.797706    8020 kubelet_node_status.go:279] Setting node annotation to enable volume controller attach/detach
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.798048    8020 setters.go:72] Using node IP: "172.16.10.36"
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.799200    8020 kubelet_node_status.go:447] Recording NodeHasSufficientMemory event message for node ctl01
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.799280    8020 kubelet_node_status.go:447] Recording NodeHasNoDiskPressure event message for node ctl01
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.799305    8020 kubelet_node_status.go:447] Recording NodeHasSufficientPID event message for node ctl01
Oct 26 13:59:43 ctl01 kubelet[8020]: I1026 13:59:43.799343    8020 kubelet_node_status.go:72] Attempting to register node ctl01
Oct 26 13:59:43 ctl01 kubelet[8020]: E1026 13:59:43.799757    8020 kubelet_node_status.go:94] Unable to register node "ctl01" with API server: Post https://172.16.10.36:443/api/v1/nodes: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:43 ctl01 kubelet[8020]: E1026 13:59:43.895317    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:43 ctl01 kubelet[8020]: E1026 13:59:43.995542    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:44 ctl01 kubelet[8020]: E1026 13:59:44.084592    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://172.16.10.36:443/api/v1/pods?fieldSelector=spec.nodeName%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:44 ctl01 kubelet[8020]: E1026 13:59:44.085099    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://172.16.10.36:443/api/v1/nodes?fieldSelector=metadata.name%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:44 ctl01 kubelet[8020]: E1026 13:59:44.086174    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:44 ctl01 kubelet[8020]: E1026 13:59:44.095704    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:44 ctl01 kubelet[8020]: E1026 13:59:44.195916    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:44 ctl01 kubelet[8020]: E1026 13:59:44.296099    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:44 ctl01 kubelet[8020]: E1026 13:59:44.396331    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:44 ctl01 kubelet[8020]: E1026 13:59:44.496595    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:44 ctl01 kubelet[8020]: E1026 13:59:44.596802    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:44 ctl01 kubelet[8020]: I1026 13:59:44.600017    8020 kubelet_node_status.go:279] Setting node annotation to enable volume controller attach/detach
Oct 26 13:59:44 ctl01 kubelet[8020]: I1026 13:59:44.600307    8020 setters.go:72] Using node IP: "172.16.10.36"
Oct 26 13:59:44 ctl01 kubelet[8020]: I1026 13:59:44.601283    8020 kubelet_node_status.go:447] Recording NodeHasSufficientMemory event message for node ctl01
Oct 26 13:59:44 ctl01 kubelet[8020]: I1026 13:59:44.601351    8020 kubelet_node_status.go:447] Recording NodeHasNoDiskPressure event message for node ctl01
Oct 26 13:59:44 ctl01 kubelet[8020]: I1026 13:59:44.601368    8020 kubelet_node_status.go:447] Recording NodeHasSufficientPID event message for node ctl01
Oct 26 13:59:44 ctl01 kubelet[8020]: I1026 13:59:44.601394    8020 kubelet_node_status.go:72] Attempting to register node ctl01
Oct 26 13:59:44 ctl01 kubelet[8020]: E1026 13:59:44.601841    8020 kubelet_node_status.go:94] Unable to register node "ctl01" with API server: Post https://172.16.10.36:443/api/v1/nodes: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:44 ctl01 kubelet[8020]: E1026 13:59:44.697100    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:44 ctl01 kubelet[8020]: E1026 13:59:44.797292    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:44 ctl01 kubelet[8020]: E1026 13:59:44.897600    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:44 ctl01 salt-minion[4526]: [INFO    ] Executing command ['tar', 'xz', '-f', '/var/cache/salt/minion/extrn_files/base/docker-prod-local.artifactory.mirantis.com/artifactory/binary-prod-local/mirantis/kubernetes/containernetworking-plugins/containernetworking-plugins_v0.7.2-173-g8db2808.tar.gz'] in directory '/opt/cni/bin/'
Oct 26 13:59:44 ctl01 kubelet[8020]: E1026 13:59:44.997821    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:45 ctl01 kubelet[8020]: E1026 13:59:45.085548    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://172.16.10.36:443/api/v1/pods?fieldSelector=spec.nodeName%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:45 ctl01 kubelet[8020]: E1026 13:59:45.088503    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://172.16.10.36:443/api/v1/nodes?fieldSelector=metadata.name%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:45 ctl01 kubelet[8020]: E1026 13:59:45.089646    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:45 ctl01 kubelet[8020]: E1026 13:59:45.098008    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:45 ctl01 kubelet[8020]: E1026 13:59:45.198197    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:45 ctl01 kubelet[8020]: E1026 13:59:45.298435    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:45 ctl01 kubelet[8020]: E1026 13:59:45.398663    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:45 ctl01 kubelet[8020]: E1026 13:59:45.498949    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:45 ctl01 salt-minion[4526]: [INFO    ] Executing command ['tar', '--version'] in directory '/root'
Oct 26 13:59:45 ctl01 salt-minion[4526]: [INFO    ] {'extracted_files': 'no tar output so far'}
Oct 26 13:59:45 ctl01 salt-minion[4526]: [INFO    ] Completed state [/opt/cni/bin] at time 13:59:45.566754 duration_in_ms=2910.198
Oct 26 13:59:45 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/proxy.kubeconfig] at time 13:59:45.567256
Oct 26 13:59:45 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/kubernetes/proxy.kubeconfig]
Oct 26 13:59:45 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/kube-proxy/proxy.kubeconfig'
Oct 26 13:59:45 ctl01 kubelet[8020]: E1026 13:59:45.599148    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:45 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:59:45 ctl01 kubelet[8020]: E1026 13:59:45.699308    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:45 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:59:45 ctl01 kubelet[8020]: E1026 13:59:45.799499    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:45 ctl01 kubelet[8020]: E1026 13:59:45.899725    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:45 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:59:45 ctl01 kubelet[8020]: E1026 13:59:45.999881    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:46 ctl01 kubelet[8020]: E1026 13:59:46.086571    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://172.16.10.36:443/api/v1/pods?fieldSelector=spec.nodeName%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:46 ctl01 kubelet[8020]: E1026 13:59:46.089449    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://172.16.10.36:443/api/v1/nodes?fieldSelector=metadata.name%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:46 ctl01 kubelet[8020]: E1026 13:59:46.090382    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:46 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:59:46 ctl01 kubelet[8020]: E1026 13:59:46.100224    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:46 ctl01 kubelet[8020]: E1026 13:59:46.200382    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:46 ctl01 kubelet[8020]: I1026 13:59:46.202147    8020 kubelet_node_status.go:279] Setting node annotation to enable volume controller attach/detach
Oct 26 13:59:46 ctl01 kubelet[8020]: I1026 13:59:46.202480    8020 setters.go:72] Using node IP: "172.16.10.36"
Oct 26 13:59:46 ctl01 kubelet[8020]: I1026 13:59:46.203591    8020 kubelet_node_status.go:447] Recording NodeHasSufficientMemory event message for node ctl01
Oct 26 13:59:46 ctl01 kubelet[8020]: I1026 13:59:46.203675    8020 kubelet_node_status.go:447] Recording NodeHasNoDiskPressure event message for node ctl01
Oct 26 13:59:46 ctl01 kubelet[8020]: I1026 13:59:46.203695    8020 kubelet_node_status.go:447] Recording NodeHasSufficientPID event message for node ctl01
Oct 26 13:59:46 ctl01 kubelet[8020]: I1026 13:59:46.203790    8020 kubelet_node_status.go:72] Attempting to register node ctl01
Oct 26 13:59:46 ctl01 kubelet[8020]: E1026 13:59:46.204217    8020 kubelet_node_status.go:94] Unable to register node "ctl01" with API server: Post https://172.16.10.36:443/api/v1/nodes: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:46 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:59:46 ctl01 salt-minion[4526]: New file
Oct 26 13:59:46 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/proxy.kubeconfig] at time 13:59:46.250529 duration_in_ms=683.271
Oct 26 13:59:46 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/systemd/system/kube-proxy.service] at time 13:59:46.251005
Oct 26 13:59:46 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/systemd/system/kube-proxy.service]
Oct 26 13:59:46 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/systemd/kube-proxy.service'
Oct 26 13:59:46 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:59:46 ctl01 salt-minion[4526]: New file
Oct 26 13:59:46 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/systemd/system/kube-proxy.service] at time 13:59:46.283311 duration_in_ms=32.305
Oct 26 13:59:46 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/default/kube-proxy] at time 13:59:46.283553
Oct 26 13:59:46 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/default/kube-proxy]
Oct 26 13:59:46 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 13:59:46 ctl01 salt-minion[4526]: New file
Oct 26 13:59:46 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/default/kube-proxy] at time 13:59:46.285822 duration_in_ms=2.269
Oct 26 13:59:46 ctl01 salt-minion[4526]: [INFO    ] Running state [kube-proxy] at time 13:59:46.287540
Oct 26 13:59:46 ctl01 salt-minion[4526]: [INFO    ] Executing state service.running for [kube-proxy]
Oct 26 13:59:46 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'status', 'kube-proxy.service', '-n', '0'] in directory '/root'
Oct 26 13:59:46 ctl01 kubelet[8020]: E1026 13:59:46.300557    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:46 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-active', 'kube-proxy.service'] in directory '/root'
Oct 26 13:59:46 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-enabled', 'kube-proxy.service'] in directory '/root'
Oct 26 13:59:46 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemd-run', '--scope', 'systemctl', 'start', 'kube-proxy.service'] in directory '/root'
Oct 26 13:59:46 ctl01 systemd[1]: Started /bin/systemctl start kube-proxy.service.
Oct 26 13:59:46 ctl01 systemd[1]: Started Kubernetes Kube-Proxy Server.
Oct 26 13:59:46 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-active', 'kube-proxy.service'] in directory '/root'
Oct 26 13:59:46 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-enabled', 'kube-proxy.service'] in directory '/root'
Oct 26 13:59:46 ctl01 kubelet[8020]: E1026 13:59:46.400698    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:46 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-enabled', 'kube-proxy.service'] in directory '/root'
Oct 26 13:59:46 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemd-run', '--scope', 'systemctl', 'enable', 'kube-proxy.service'] in directory '/root'
Oct 26 13:59:46 ctl01 systemd[1]: Started /bin/systemctl enable kube-proxy.service.
Oct 26 13:59:46 ctl01 systemd[1]: Reloading.
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499297    8230 flags.go:33] FLAG: --alsologtostderr="false"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499495    8230 flags.go:33] FLAG: --application-metrics-count-limit="100"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499522    8230 flags.go:33] FLAG: --azure-container-registry-config=""
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499533    8230 flags.go:33] FLAG: --bind-address="0.0.0.0"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499544    8230 flags.go:33] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499553    8230 flags.go:33] FLAG: --cleanup="false"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499563    8230 flags.go:33] FLAG: --cleanup-iptables="false"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499572    8230 flags.go:33] FLAG: --cleanup-ipvs="true"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499579    8230 flags.go:33] FLAG: --cloud-provider-gce-lb-src-cidrs="130.211.0.0/22,209.85.152.0/22,209.85.204.0/22,35.191.0.0/16"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499593    8230 flags.go:33] FLAG: --cluster-cidr="192.168.0.0/16"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499601    8230 flags.go:33] FLAG: --config=""
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499609    8230 flags.go:33] FLAG: --config-sync-period="15m0s"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499620    8230 flags.go:33] FLAG: --conntrack-max="0"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499630    8230 flags.go:33] FLAG: --conntrack-max-per-core="32768"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499639    8230 flags.go:33] FLAG: --conntrack-min="131072"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499647    8230 flags.go:33] FLAG: --conntrack-tcp-timeout-close-wait="1h0m0s"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499656    8230 flags.go:33] FLAG: --conntrack-tcp-timeout-established="24h0m0s"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499664    8230 flags.go:33] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499673    8230 flags.go:33] FLAG: --containerd="unix:///var/run/containerd.sock"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499682    8230 flags.go:33] FLAG: --default-not-ready-toleration-seconds="300"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499690    8230 flags.go:33] FLAG: --default-unreachable-toleration-seconds="300"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499698    8230 flags.go:33] FLAG: --docker="unix:///var/run/docker.sock"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499706    8230 flags.go:33] FLAG: --docker-env-metadata-whitelist=""
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499713    8230 flags.go:33] FLAG: --docker-only="false"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499721    8230 flags.go:33] FLAG: --docker-root="/var/lib/docker"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499729    8230 flags.go:33] FLAG: --docker-tls="false"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499737    8230 flags.go:33] FLAG: --docker-tls-ca="ca.pem"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499745    8230 flags.go:33] FLAG: --docker-tls-cert="cert.pem"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499753    8230 flags.go:33] FLAG: --docker-tls-key="key.pem"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499760    8230 flags.go:33] FLAG: --enable-load-reader="false"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499768    8230 flags.go:33] FLAG: --event-storage-age-limit="default=0"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499776    8230 flags.go:33] FLAG: --event-storage-event-limit="default=0"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499784    8230 flags.go:33] FLAG: --feature-gates=""
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499796    8230 flags.go:33] FLAG: --global-housekeeping-interval="1m0s"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499816    8230 flags.go:33] FLAG: --healthz-bind-address="0.0.0.0:10256"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499825    8230 flags.go:33] FLAG: --healthz-port="10256"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499833    8230 flags.go:33] FLAG: --help="false"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499841    8230 flags.go:33] FLAG: --hostname-override=""
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499849    8230 flags.go:33] FLAG: --housekeeping-interval="10s"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499857    8230 flags.go:33] FLAG: --iptables-masquerade-bit="14"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499877    8230 flags.go:33] FLAG: --iptables-min-sync-period="0s"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499884    8230 flags.go:33] FLAG: --iptables-sync-period="30s"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499889    8230 flags.go:33] FLAG: --ipvs-exclude-cidrs="[]"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499899    8230 flags.go:33] FLAG: --ipvs-min-sync-period="0s"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499905    8230 flags.go:33] FLAG: --ipvs-scheduler=""
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499910    8230 flags.go:33] FLAG: --ipvs-sync-period="30s"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499916    8230 flags.go:33] FLAG: --kube-api-burst="10"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499922    8230 flags.go:33] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499929    8230 flags.go:33] FLAG: --kube-api-qps="5"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499937    8230 flags.go:33] FLAG: --kubeconfig="/etc/kubernetes/proxy.kubeconfig"
Oct 26 13:59:46 ctl01 kubelet[8020]: E1026 13:59:46.500877    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499944    8230 flags.go:33] FLAG: --log-backtrace-at=":0"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499952    8230 flags.go:33] FLAG: --log-cadvisor-usage="false"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499959    8230 flags.go:33] FLAG: --log-dir=""
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499964    8230 flags.go:33] FLAG: --log-file=""
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499969    8230 flags.go:33] FLAG: --log-flush-frequency="5s"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499975    8230 flags.go:33] FLAG: --logtostderr="true"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499981    8230 flags.go:33] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499988    8230 flags.go:33] FLAG: --masquerade-all="false"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499993    8230 flags.go:33] FLAG: --master=""
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.499998    8230 flags.go:33] FLAG: --mesos-agent="127.0.0.1:5051"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.500004    8230 flags.go:33] FLAG: --mesos-agent-timeout="10s"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.500010    8230 flags.go:33] FLAG: --metrics-bind-address="127.0.0.1:10249"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.500016    8230 flags.go:33] FLAG: --metrics-port="10249"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.500022    8230 flags.go:33] FLAG: --nodeport-addresses="[]"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.500032    8230 flags.go:33] FLAG: --oom-score-adj="-999"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.500038    8230 flags.go:33] FLAG: --profiling="false"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.500053    8230 flags.go:33] FLAG: --proxy-mode="iptables"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.500061    8230 flags.go:33] FLAG: --proxy-port-range=""
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.500067    8230 flags.go:33] FLAG: --resource-container="/kube-proxy"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.500073    8230 flags.go:33] FLAG: --skip-headers="false"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.500079    8230 flags.go:33] FLAG: --stderrthreshold="2"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.500084    8230 flags.go:33] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.500090    8230 flags.go:33] FLAG: --storage-driver-db="cadvisor"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.500096    8230 flags.go:33] FLAG: --storage-driver-host="localhost:8086"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.500102    8230 flags.go:33] FLAG: --storage-driver-password="root"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.500107    8230 flags.go:33] FLAG: --storage-driver-secure="false"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.500112    8230 flags.go:33] FLAG: --storage-driver-table="stats"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.500118    8230 flags.go:33] FLAG: --storage-driver-user="root"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.500123    8230 flags.go:33] FLAG: --udp-timeout="250ms"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.500129    8230 flags.go:33] FLAG: --v="2"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.500135    8230 flags.go:33] FLAG: --version="false"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.500143    8230 flags.go:33] FLAG: --vmodule=""
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.500149    8230 flags.go:33] FLAG: --write-config-to=""
Oct 26 13:59:46 ctl01 kube-proxy[8230]: W1026 13:59:46.500158    8230 server.go:198] WARNING: all flags other than --config, --write-config-to, and --cleanup are deprecated. Please begin using a config file ASAP.
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.500201    8230 feature_gate.go:206] feature gates: &{map[]}
Oct 26 13:59:46 ctl01 kernel: [  184.201769] IPVS: Registered protocols (TCP, UDP, SCTP, AH, ESP)
Oct 26 13:59:46 ctl01 kernel: [  184.201813] IPVS: Connection hash table configured (size=4096, memory=64Kbytes)
Oct 26 13:59:46 ctl01 systemd[1]: kubelet.service: Dependency Conflicts=cadvisor.service dropped, merged into kubelet.service
Oct 26 13:59:46 ctl01 systemd[1]: kubelet.service: Dependency ConflictedBy=cadvisor.service dropped, merged into kubelet.service
Oct 26 13:59:46 ctl01 systemd[1]: Started ACPI event daemon.
Oct 26 13:59:46 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-enabled', 'kube-proxy.service'] in directory '/root'
Oct 26 13:59:46 ctl01 salt-minion[4526]: [INFO    ] {'kube-proxy': True}
Oct 26 13:59:46 ctl01 salt-minion[4526]: [INFO    ] Completed state [kube-proxy] at time 13:59:46.569534 duration_in_ms=281.984
Oct 26 13:59:46 ctl01 salt-minion[4526]: [INFO    ] Returning information for job: 20191026135851446315
Oct 26 13:59:46 ctl01 kubelet[8020]: E1026 13:59:46.601069    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:46 ctl01 kubelet[8020]: E1026 13:59:46.701346    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:46 ctl01 kernel: [  184.447910] IPVS: ipvs loaded.
Oct 26 13:59:46 ctl01 kernel: [  184.452165] IPVS: [rr] scheduler registered.
Oct 26 13:59:46 ctl01 kernel: [  184.455130] IPVS: [wrr] scheduler registered.
Oct 26 13:59:46 ctl01 kernel: [  184.458312] IPVS: [sh] scheduler registered.
Oct 26 13:59:46 ctl01 kube-proxy[8230]: W1026 13:59:46.786382    8230 node.go:103] Failed to retrieve node info: Get https://172.16.10.36:443/api/v1/nodes/ctl01: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.786457    8230 server_others.go:148] Using iptables Proxier.
Oct 26 13:59:46 ctl01 kube-proxy[8230]: W1026 13:59:46.786683    8230 proxier.go:314] invalid nodeIP, initializing kube-proxy with 127.0.0.1 as nodeIP
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.786773    8230 server_others.go:178] Tearing down inactive rules.
Oct 26 13:59:46 ctl01 kubelet[8020]: E1026 13:59:46.801624    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.809677    8230 server.go:483] Version: v1.13.5-3+98374c02d2d8c1
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.818160    8230 server.go:509] Running in resource-only container "/kube-proxy"
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.819509    8230 conntrack.go:100] Set sysctl 'net/netfilter/nf_conntrack_max' to 262144
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.819572    8230 conntrack.go:52] Setting nf_conntrack_max to 262144
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.819752    8230 conntrack.go:100] Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_established' to 86400
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.819825    8230 conntrack.go:100] Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_close_wait' to 3600
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.820048    8230 config.go:102] Starting endpoints config controller
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.820081    8230 controller_utils.go:1027] Waiting for caches to sync for endpoints config controller
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.820063    8230 config.go:202] Starting service config controller
Oct 26 13:59:46 ctl01 kube-proxy[8230]: I1026 13:59:46.820129    8230 controller_utils.go:1027] Waiting for caches to sync for service config controller
Oct 26 13:59:46 ctl01 kube-proxy[8230]: E1026 13:59:46.820793    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:46 ctl01 kube-proxy[8230]: E1026 13:59:46.820816    8230 event.go:212] Unable to write event: 'Post https://172.16.10.36:443/api/v1/namespaces/default/events: dial tcp 172.16.10.36:443: connect: connection refused' (may retry after sleeping)
Oct 26 13:59:46 ctl01 kube-proxy[8230]: E1026 13:59:46.820878    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: Get https://172.16.10.36:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:46 ctl01 kubelet[8020]: E1026 13:59:46.901802    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:47 ctl01 kubelet[8020]: E1026 13:59:47.001995    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:47 ctl01 kubelet[8020]: E1026 13:59:47.088533    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://172.16.10.36:443/api/v1/pods?fieldSelector=spec.nodeName%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:47 ctl01 kubelet[8020]: E1026 13:59:47.090551    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://172.16.10.36:443/api/v1/nodes?fieldSelector=metadata.name%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:47 ctl01 kubelet[8020]: E1026 13:59:47.091601    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:47 ctl01 kubelet[8020]: E1026 13:59:47.102230    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:47 ctl01 kubelet[8020]: E1026 13:59:47.202597    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:47 ctl01 salt-minion[4526]: [INFO    ] User sudo_ubuntu Executing command cmd.run with jid 20191026135947236400
Oct 26 13:59:47 ctl01 salt-minion[4526]: [INFO    ] Starting a new job with PID 8347
Oct 26 13:59:47 ctl01 salt-minion[4526]: [INFO    ] Executing command 'calicoctl node status' in directory '/root'
Oct 26 13:59:47 ctl01 kubelet[8020]: E1026 13:59:47.302797    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:47 ctl01 salt-minion[4526]: [INFO    ] Returning information for job: 20191026135947236400
Oct 26 13:59:47 ctl01 kubelet[8020]: E1026 13:59:47.403073    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:47 ctl01 kubelet[8020]: E1026 13:59:47.503386    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:47 ctl01 kubelet[8020]: E1026 13:59:47.603621    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:47 ctl01 kubelet[8020]: E1026 13:59:47.703864    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:47 ctl01 kubelet[8020]: E1026 13:59:47.804067    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:47 ctl01 kube-proxy[8230]: E1026 13:59:47.821960    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:47 ctl01 kube-proxy[8230]: E1026 13:59:47.822683    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: Get https://172.16.10.36:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:47 ctl01 kubelet[8020]: E1026 13:59:47.904454    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:47 ctl01 salt-minion[4526]: [INFO    ] User sudo_ubuntu Executing command cmd.run with jid 20191026135947953882
Oct 26 13:59:47 ctl01 salt-minion[4526]: [INFO    ] Starting a new job with PID 8368
Oct 26 13:59:47 ctl01 salt-minion[4526]: [INFO    ] Executing command 'calicoctl get ippool' in directory '/root'
Oct 26 13:59:48 ctl01 kubelet[8020]: E1026 13:59:48.004693    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:48 ctl01 kubelet[8020]: E1026 13:59:48.089617    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://172.16.10.36:443/api/v1/pods?fieldSelector=spec.nodeName%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:48 ctl01 kubelet[8020]: E1026 13:59:48.091199    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://172.16.10.36:443/api/v1/nodes?fieldSelector=metadata.name%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:48 ctl01 kubelet[8020]: E1026 13:59:48.092266    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:48 ctl01 kubelet[8020]: E1026 13:59:48.104876    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:48 ctl01 salt-minion[4526]: [INFO    ] Returning information for job: 20191026135947953882
Oct 26 13:59:48 ctl01 kubelet[8020]: E1026 13:59:48.205273    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:48 ctl01 kubelet[8020]: E1026 13:59:48.248628    8020 event.go:212] Unable to write event: 'Post https://172.16.10.36:443/api/v1/namespaces/default/events: dial tcp 172.16.10.36:443: connect: connection refused' (may retry after sleeping)
Oct 26 13:59:48 ctl01 kubelet[8020]: E1026 13:59:48.305474    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:48 ctl01 kubelet[8020]: E1026 13:59:48.405604    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:48 ctl01 kubelet[8020]: E1026 13:59:48.505991    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:48 ctl01 kubelet[8020]: E1026 13:59:48.606249    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:48 ctl01 kubelet[8020]: E1026 13:59:48.706583    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:48 ctl01 salt-minion[4526]: [INFO    ] User sudo_ubuntu Executing command state.sls with jid 20191026135948738320
Oct 26 13:59:48 ctl01 salt-minion[4526]: [INFO    ] Starting a new job with PID 8387
Oct 26 13:59:48 ctl01 kubelet[8020]: E1026 13:59:48.806873    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:48 ctl01 kube-proxy[8230]: E1026 13:59:48.823250    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:48 ctl01 kube-proxy[8230]: E1026 13:59:48.823726    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: Get https://172.16.10.36:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:48 ctl01 kubelet[8020]: E1026 13:59:48.907028    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:49 ctl01 kubelet[8020]: E1026 13:59:49.007214    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:49 ctl01 kubelet[8020]: E1026 13:59:49.090631    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://172.16.10.36:443/api/v1/pods?fieldSelector=spec.nodeName%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:49 ctl01 kubelet[8020]: E1026 13:59:49.092056    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://172.16.10.36:443/api/v1/nodes?fieldSelector=metadata.name%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:49 ctl01 kubelet[8020]: E1026 13:59:49.092889    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:49 ctl01 kubelet[8020]: E1026 13:59:49.107510    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:49 ctl01 kubelet[8020]: E1026 13:59:49.207759    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:49 ctl01 kubelet[8020]: E1026 13:59:49.307954    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:49 ctl01 kubelet[8020]: I1026 13:59:49.404442    8020 kubelet_node_status.go:279] Setting node annotation to enable volume controller attach/detach
Oct 26 13:59:49 ctl01 kubelet[8020]: I1026 13:59:49.404779    8020 setters.go:72] Using node IP: "172.16.10.36"
Oct 26 13:59:49 ctl01 kubelet[8020]: I1026 13:59:49.405740    8020 kubelet_node_status.go:447] Recording NodeHasSufficientMemory event message for node ctl01
Oct 26 13:59:49 ctl01 kubelet[8020]: I1026 13:59:49.405778    8020 kubelet_node_status.go:447] Recording NodeHasNoDiskPressure event message for node ctl01
Oct 26 13:59:49 ctl01 kubelet[8020]: I1026 13:59:49.405809    8020 kubelet_node_status.go:447] Recording NodeHasSufficientPID event message for node ctl01
Oct 26 13:59:49 ctl01 kubelet[8020]: I1026 13:59:49.405839    8020 kubelet_node_status.go:72] Attempting to register node ctl01
Oct 26 13:59:49 ctl01 kubelet[8020]: E1026 13:59:49.406251    8020 kubelet_node_status.go:94] Unable to register node "ctl01" with API server: Post https://172.16.10.36:443/api/v1/nodes: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:49 ctl01 kubelet[8020]: E1026 13:59:49.408122    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:49 ctl01 kubelet[8020]: E1026 13:59:49.508245    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:49 ctl01 kubelet[8020]: E1026 13:59:49.608418    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:49 ctl01 kubelet[8020]: E1026 13:59:49.708609    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:49 ctl01 kubelet[8020]: E1026 13:59:49.808783    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:49 ctl01 kube-proxy[8230]: E1026 13:59:49.825181    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: Get https://172.16.10.36:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:49 ctl01 kube-proxy[8230]: E1026 13:59:49.834992    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:49 ctl01 kubelet[8020]: E1026 13:59:49.908977    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:50 ctl01 kubelet[8020]: E1026 13:59:50.009146    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:50 ctl01 kubelet[8020]: E1026 13:59:50.091468    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://172.16.10.36:443/api/v1/pods?fieldSelector=spec.nodeName%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:50 ctl01 kubelet[8020]: E1026 13:59:50.092640    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://172.16.10.36:443/api/v1/nodes?fieldSelector=metadata.name%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:50 ctl01 kubelet[8020]: E1026 13:59:50.093729    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:50 ctl01 kubelet[8020]: E1026 13:59:50.109257    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:50 ctl01 kubelet[8020]: E1026 13:59:50.209437    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:50 ctl01 kubelet[8020]: E1026 13:59:50.309698    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:50 ctl01 kubelet[8020]: E1026 13:59:50.409973    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:50 ctl01 kubelet[8020]: E1026 13:59:50.510184    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:50 ctl01 kubelet[8020]: E1026 13:59:50.610440    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:50 ctl01 kubelet[8020]: E1026 13:59:50.710681    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:50 ctl01 kubelet[8020]: E1026 13:59:50.810875    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:50 ctl01 kube-proxy[8230]: E1026 13:59:50.826018    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: Get https://172.16.10.36:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:50 ctl01 kube-proxy[8230]: E1026 13:59:50.835738    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:50 ctl01 kubelet[8020]: E1026 13:59:50.911111    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:51 ctl01 kubelet[8020]: E1026 13:59:51.011394    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:51 ctl01 kubelet[8020]: E1026 13:59:51.092011    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://172.16.10.36:443/api/v1/pods?fieldSelector=spec.nodeName%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:51 ctl01 kubelet[8020]: E1026 13:59:51.093375    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://172.16.10.36:443/api/v1/nodes?fieldSelector=metadata.name%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:51 ctl01 kubelet[8020]: E1026 13:59:51.094228    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:51 ctl01 kubelet[8020]: E1026 13:59:51.111623    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:51 ctl01 kubelet[8020]: E1026 13:59:51.211798    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:51 ctl01 kubelet[8020]: E1026 13:59:51.311947    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:51 ctl01 kubelet[8020]: E1026 13:59:51.412128    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:51 ctl01 kubelet[8020]: E1026 13:59:51.512254    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:51 ctl01 kubelet[8020]: E1026 13:59:51.612403    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:51 ctl01 kubelet[8020]: E1026 13:59:51.712473    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:51 ctl01 kubelet[8020]: E1026 13:59:51.812688    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:51 ctl01 kube-proxy[8230]: E1026 13:59:51.826626    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: Get https://172.16.10.36:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:51 ctl01 kube-proxy[8230]: E1026 13:59:51.836466    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:51 ctl01 kubelet[8020]: E1026 13:59:51.912864    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:52 ctl01 kubelet[8020]: E1026 13:59:52.013081    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:52 ctl01 kubelet[8020]: E1026 13:59:52.092654    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://172.16.10.36:443/api/v1/pods?fieldSelector=spec.nodeName%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:52 ctl01 kubelet[8020]: E1026 13:59:52.094045    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://172.16.10.36:443/api/v1/nodes?fieldSelector=metadata.name%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:52 ctl01 kubelet[8020]: E1026 13:59:52.095099    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:52 ctl01 kubelet[8020]: E1026 13:59:52.113202    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:52 ctl01 kubelet[8020]: E1026 13:59:52.213333    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:52 ctl01 kubelet[8020]: E1026 13:59:52.313516    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:52 ctl01 kubelet[8020]: E1026 13:59:52.413700    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:52 ctl01 kubelet[8020]: E1026 13:59:52.513976    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:52 ctl01 kubelet[8020]: E1026 13:59:52.614178    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:52 ctl01 kubelet[8020]: E1026 13:59:52.714407    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:52 ctl01 kubelet[8020]: E1026 13:59:52.814622    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:52 ctl01 kube-proxy[8230]: E1026 13:59:52.827505    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: Get https://172.16.10.36:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:52 ctl01 kube-proxy[8230]: E1026 13:59:52.837036    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:52 ctl01 kubelet[8020]: E1026 13:59:52.914779    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:53 ctl01 kubelet[8020]: E1026 13:59:53.014893    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:53 ctl01 kubelet[8020]: E1026 13:59:53.093257    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://172.16.10.36:443/api/v1/pods?fieldSelector=spec.nodeName%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:53 ctl01 kubelet[8020]: E1026 13:59:53.094614    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://172.16.10.36:443/api/v1/nodes?fieldSelector=metadata.name%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:53 ctl01 kubelet[8020]: E1026 13:59:53.095620    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:53 ctl01 kubelet[8020]: E1026 13:59:53.115105    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:53 ctl01 kubelet[8020]: E1026 13:59:53.215275    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:53 ctl01 kubelet[8020]: E1026 13:59:53.224759    8020 eviction_manager.go:247] eviction manager: failed to get summary stats: failed to get node info: node "ctl01" not found
Oct 26 13:59:53 ctl01 kubelet[8020]: E1026 13:59:53.315436    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:53 ctl01 kubelet[8020]: E1026 13:59:53.415583    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:53 ctl01 kubelet[8020]: E1026 13:59:53.515715    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:53 ctl01 kubelet[8020]: E1026 13:59:53.615857    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:53 ctl01 kubelet[8020]: E1026 13:59:53.716009    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:53 ctl01 kubelet[8020]: E1026 13:59:53.816148    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:53 ctl01 kube-proxy[8230]: E1026 13:59:53.828138    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: Get https://172.16.10.36:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:53 ctl01 kube-proxy[8230]: E1026 13:59:53.837549    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:53 ctl01 kubelet[8020]: E1026 13:59:53.916388    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:54 ctl01 kubelet[8020]: E1026 13:59:54.016576    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:54 ctl01 kubelet[8020]: E1026 13:59:54.093759    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://172.16.10.36:443/api/v1/pods?fieldSelector=spec.nodeName%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:54 ctl01 kubelet[8020]: E1026 13:59:54.095024    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://172.16.10.36:443/api/v1/nodes?fieldSelector=metadata.name%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:54 ctl01 kubelet[8020]: E1026 13:59:54.096059    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:54 ctl01 kubelet[8020]: E1026 13:59:54.116715    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:54 ctl01 kubelet[8020]: E1026 13:59:54.216942    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:54 ctl01 kube-proxy[8230]: E1026 13:59:54.294148    8230 event.go:212] Unable to write event: 'Post https://172.16.10.36:443/api/v1/namespaces/default/events: dial tcp 172.16.10.36:443: connect: connection refused' (may retry after sleeping)
Oct 26 13:59:54 ctl01 kubelet[8020]: E1026 13:59:54.317126    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:54 ctl01 kubelet[8020]: E1026 13:59:54.417341    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:54 ctl01 kubelet[8020]: E1026 13:59:54.517557    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:54 ctl01 kubelet[8020]: E1026 13:59:54.617717    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:54 ctl01 kubelet[8020]: E1026 13:59:54.717851    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:54 ctl01 kubelet[8020]: E1026 13:59:54.818025    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:54 ctl01 kube-proxy[8230]: E1026 13:59:54.829050    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: Get https://172.16.10.36:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:54 ctl01 kube-proxy[8230]: E1026 13:59:54.869668    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:54 ctl01 kubelet[8020]: E1026 13:59:54.918197    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:55 ctl01 kubelet[8020]: E1026 13:59:55.018382    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:55 ctl01 kubelet[8020]: E1026 13:59:55.094396    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://172.16.10.36:443/api/v1/pods?fieldSelector=spec.nodeName%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:55 ctl01 kubelet[8020]: E1026 13:59:55.095615    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://172.16.10.36:443/api/v1/nodes?fieldSelector=metadata.name%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:55 ctl01 kubelet[8020]: E1026 13:59:55.096629    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:55 ctl01 salt-minion[4526]: [INFO    ] Loading fresh modules for state activity
Oct 26 13:59:55 ctl01 kubelet[8020]: E1026 13:59:55.118650    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:55 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'etcd/server/setup.sls'
Oct 26 13:59:55 ctl01 salt-minion[4526]: [INFO    ] Running state [/calico/ipam/v2/assignment/ipv4/block/192.168.0.0-16] at time 13:59:55.191156
Oct 26 13:59:55 ctl01 salt-minion[4526]: [INFO    ] Executing state etcd.set for [/calico/ipam/v2/assignment/ipv4/block/192.168.0.0-16]
Oct 26 13:59:55 ctl01 salt-minion[4526]: [INFO    ] Starting new HTTP connection (1): 127.0.0.1
Oct 26 13:59:55 ctl01 salt-minion[4526]: [ERROR   ] etcd: Key not found : /calico
Oct 26 13:59:55 ctl01 salt-minion[4526]: [INFO    ] Starting new HTTP connection (1): 127.0.0.1
Oct 26 13:59:55 ctl01 salt-minion[4526]: [INFO    ] {'/calico/ipam/v2/assignment/ipv4/block/192.168.0.0-16': '{"masquerade":true,"cidr":"192.168.0.0/16"}'}
Oct 26 13:59:55 ctl01 salt-minion[4526]: [INFO    ] Completed state [/calico/ipam/v2/assignment/ipv4/block/192.168.0.0-16] at time 13:59:55.199392 duration_in_ms=8.236
Oct 26 13:59:55 ctl01 salt-minion[4526]: [INFO    ] Returning information for job: 20191026135948738320
Oct 26 13:59:55 ctl01 kubelet[8020]: E1026 13:59:55.218917    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:55 ctl01 kubelet[8020]: E1026 13:59:55.319199    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:55 ctl01 kubelet[8020]: E1026 13:59:55.419408    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:55 ctl01 kubelet[8020]: E1026 13:59:55.519680    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:55 ctl01 kubelet[8020]: E1026 13:59:55.619880    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:55 ctl01 kubelet[8020]: E1026 13:59:55.720154    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:55 ctl01 kubelet[8020]: I1026 13:59:55.806494    8020 kubelet_node_status.go:279] Setting node annotation to enable volume controller attach/detach
Oct 26 13:59:55 ctl01 kubelet[8020]: I1026 13:59:55.807103    8020 setters.go:72] Using node IP: "172.16.10.36"
Oct 26 13:59:55 ctl01 kubelet[8020]: I1026 13:59:55.809081    8020 kubelet_node_status.go:447] Recording NodeHasSufficientMemory event message for node ctl01
Oct 26 13:59:55 ctl01 kubelet[8020]: I1026 13:59:55.809169    8020 kubelet_node_status.go:447] Recording NodeHasNoDiskPressure event message for node ctl01
Oct 26 13:59:55 ctl01 kubelet[8020]: I1026 13:59:55.809200    8020 kubelet_node_status.go:447] Recording NodeHasSufficientPID event message for node ctl01
Oct 26 13:59:55 ctl01 kubelet[8020]: I1026 13:59:55.809244    8020 kubelet_node_status.go:72] Attempting to register node ctl01
Oct 26 13:59:55 ctl01 kubelet[8020]: E1026 13:59:55.809834    8020 kubelet_node_status.go:94] Unable to register node "ctl01" with API server: Post https://172.16.10.36:443/api/v1/nodes: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:55 ctl01 salt-minion[4526]: [INFO    ] User sudo_ubuntu Executing command state.sls with jid 20191026135955807384
Oct 26 13:59:55 ctl01 kubelet[8020]: E1026 13:59:55.820409    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:55 ctl01 kube-proxy[8230]: E1026 13:59:55.829868    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: Get https://172.16.10.36:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:55 ctl01 salt-minion[4526]: [INFO    ] Starting a new job with PID 8422
Oct 26 13:59:55 ctl01 kube-proxy[8230]: E1026 13:59:55.870339    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:55 ctl01 kubelet[8020]: E1026 13:59:55.920624    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:56 ctl01 kubelet[8020]: E1026 13:59:56.020775    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:56 ctl01 kubelet[8020]: E1026 13:59:56.095131    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://172.16.10.36:443/api/v1/pods?fieldSelector=spec.nodeName%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:56 ctl01 kubelet[8020]: E1026 13:59:56.096242    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://172.16.10.36:443/api/v1/nodes?fieldSelector=metadata.name%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:56 ctl01 kubelet[8020]: E1026 13:59:56.097212    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:56 ctl01 kubelet[8020]: E1026 13:59:56.121022    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:56 ctl01 kubelet[8020]: E1026 13:59:56.221262    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:56 ctl01 kubelet[8020]: E1026 13:59:56.321505    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:56 ctl01 kubelet[8020]: E1026 13:59:56.421693    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:56 ctl01 kubelet[8020]: E1026 13:59:56.521847    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:56 ctl01 kubelet[8020]: E1026 13:59:56.621991    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:56 ctl01 kubelet[8020]: E1026 13:59:56.722133    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:56 ctl01 kubelet[8020]: E1026 13:59:56.822360    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:56 ctl01 kube-proxy[8230]: E1026 13:59:56.830604    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: Get https://172.16.10.36:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:56 ctl01 kube-proxy[8230]: E1026 13:59:56.871560    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:56 ctl01 salt-minion[4526]: [INFO    ] Loading fresh modules for state activity
Oct 26 13:59:56 ctl01 kubelet[8020]: E1026 13:59:56.922520    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:56 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/init.sls'
Oct 26 13:59:57 ctl01 kubelet[8020]: E1026 13:59:57.022746    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:57 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/master/init.sls'
Oct 26 13:59:57 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:59:57 ctl01 kubelet[8020]: E1026 13:59:57.095916    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://172.16.10.36:443/api/v1/pods?fieldSelector=spec.nodeName%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:57 ctl01 kubelet[8020]: E1026 13:59:57.096934    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://172.16.10.36:443/api/v1/nodes?fieldSelector=metadata.name%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:57 ctl01 kubelet[8020]: E1026 13:59:57.098022    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:57 ctl01 kubelet[8020]: E1026 13:59:57.122950    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:57 ctl01 kubelet[8020]: E1026 13:59:57.223112    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:57 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:59:57 ctl01 kubelet[8020]: E1026 13:59:57.323251    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:57 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/master/service.sls'
Oct 26 13:59:57 ctl01 kubelet[8020]: E1026 13:59:57.423440    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:57 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:59:57 ctl01 kubelet[8020]: E1026 13:59:57.523640    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:57 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:59:57 ctl01 kubelet[8020]: E1026 13:59:57.623961    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:57 ctl01 kubelet[8020]: E1026 13:59:57.724170    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:57 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:59:57 ctl01 kubelet[8020]: E1026 13:59:57.824328    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:57 ctl01 kube-proxy[8230]: E1026 13:59:57.831595    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: Get https://172.16.10.36:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:57 ctl01 kube-proxy[8230]: E1026 13:59:57.872312    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:57 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:59:57 ctl01 kubelet[8020]: E1026 13:59:57.924513    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:58 ctl01 kubelet[8020]: E1026 13:59:58.024756    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:58 ctl01 kubelet[8020]: E1026 13:59:58.096569    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://172.16.10.36:443/api/v1/pods?fieldSelector=spec.nodeName%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:58 ctl01 kubelet[8020]: E1026 13:59:58.097365    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://172.16.10.36:443/api/v1/nodes?fieldSelector=metadata.name%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:58 ctl01 kubelet[8020]: E1026 13:59:58.098469    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:58 ctl01 kubelet[8020]: E1026 13:59:58.124902    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:58 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:59:58 ctl01 kubelet[8020]: E1026 13:59:58.225036    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:58 ctl01 kubelet[8020]: E1026 13:59:58.249455    8020 event.go:212] Unable to write event: 'Post https://172.16.10.36:443/api/v1/namespaces/default/events: dial tcp 172.16.10.36:443: connect: connection refused' (may retry after sleeping)
Oct 26 13:59:58 ctl01 kubelet[8020]: E1026 13:59:58.325244    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:58 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:59:58 ctl01 kubelet[8020]: E1026 13:59:58.425418    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:58 ctl01 kubelet[8020]: E1026 13:59:58.525597    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:58 ctl01 kubelet[8020]: E1026 13:59:58.625791    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:58 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:59:58 ctl01 kubelet[8020]: E1026 13:59:58.726054    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:58 ctl01 kubelet[8020]: E1026 13:59:58.826262    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:58 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:59:58 ctl01 kube-proxy[8230]: E1026 13:59:58.832542    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: Get https://172.16.10.36:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:58 ctl01 kube-proxy[8230]: E1026 13:59:58.872972    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:58 ctl01 kubelet[8020]: E1026 13:59:58.926430    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:58 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:59:59 ctl01 kubelet[8020]: E1026 13:59:59.026611    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:59 ctl01 kubelet[8020]: E1026 13:59:59.097584    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://172.16.10.36:443/api/v1/pods?fieldSelector=spec.nodeName%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:59 ctl01 kubelet[8020]: E1026 13:59:59.098333    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://172.16.10.36:443/api/v1/nodes?fieldSelector=metadata.name%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:59 ctl01 kubelet[8020]: E1026 13:59:59.099215    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:59 ctl01 kubelet[8020]: E1026 13:59:59.126735    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:59 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:59:59 ctl01 kubelet[8020]: E1026 13:59:59.226904    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:59 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:59:59 ctl01 kubelet[8020]: E1026 13:59:59.327136    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:59 ctl01 kubelet[8020]: E1026 13:59:59.427318    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:59 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 13:59:59 ctl01 kubelet[8020]: E1026 13:59:59.527488    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:59 ctl01 kubelet[8020]: E1026 13:59:59.627649    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:59 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/master/controller.sls'
Oct 26 13:59:59 ctl01 kubelet[8020]: E1026 13:59:59.727824    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:59 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 13:59:59 ctl01 kubelet[8020]: E1026 13:59:59.828055    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:59 ctl01 kube-proxy[8230]: E1026 13:59:59.833410    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: Get https://172.16.10.36:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:59 ctl01 kube-proxy[8230]: E1026 13:59:59.873664    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 13:59:59 ctl01 kubelet[8020]: E1026 13:59:59.928230    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 13:59:59 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:00 ctl01 kubelet[8020]: E1026 14:00:00.028410    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:00 ctl01 kubelet[8020]: E1026 14:00:00.098634    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://172.16.10.36:443/api/v1/pods?fieldSelector=spec.nodeName%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:00 ctl01 kubelet[8020]: E1026 14:00:00.099532    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://172.16.10.36:443/api/v1/nodes?fieldSelector=metadata.name%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:00 ctl01 kubelet[8020]: E1026 14:00:00.100615    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:00 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:00 ctl01 kubelet[8020]: E1026 14:00:00.128556    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:00 ctl01 kubelet[8020]: E1026 14:00:00.228706    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:00 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:00 ctl01 kubelet[8020]: E1026 14:00:00.328876    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:00 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:00 ctl01 kubelet[8020]: E1026 14:00:00.429059    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:00 ctl01 kubelet[8020]: E1026 14:00:00.529252    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:00 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:00 ctl01 kubelet[8020]: E1026 14:00:00.629505    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:00 ctl01 salt-minion[4526]: [INFO    ] Executing command 'date "+%FT%TZ"' in directory '/root'
Oct 26 14:00:00 ctl01 kubelet[8020]: E1026 14:00:00.729672    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:00 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/master/setup.sls'
Oct 26 14:00:00 ctl01 kubelet[8020]: E1026 14:00:00.829840    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:00 ctl01 kube-proxy[8230]: E1026 14:00:00.834777    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: Get https://172.16.10.36:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:00 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:00 ctl01 kube-proxy[8230]: E1026 14:00:00.874416    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:00 ctl01 kubelet[8020]: E1026 14:00:00.930007    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:01 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:01 ctl01 kubelet[8020]: E1026 14:00:01.030269    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:01 ctl01 kubelet[8020]: E1026 14:00:01.099621    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://172.16.10.36:443/api/v1/pods?fieldSelector=spec.nodeName%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:01 ctl01 kubelet[8020]: E1026 14:00:01.100580    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://172.16.10.36:443/api/v1/nodes?fieldSelector=metadata.name%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:01 ctl01 kubelet[8020]: E1026 14:00:01.101788    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:01 ctl01 kubelet[8020]: E1026 14:00:01.130486    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:01 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:01 ctl01 kubelet[8020]: E1026 14:00:01.230775    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:01 ctl01 kubelet[8020]: E1026 14:00:01.331045    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:01 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:01 ctl01 kubelet[8020]: E1026 14:00:01.431198    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:01 ctl01 kubelet[8020]: E1026 14:00:01.531411    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:01 ctl01 kubelet[8020]: E1026 14:00:01.631602    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:01 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:01 ctl01 kubelet[8020]: E1026 14:00:01.731865    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:01 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:01 ctl01 kubelet[8020]: E1026 14:00:01.832087    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:01 ctl01 kube-proxy[8230]: E1026 14:00:01.835851    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: Get https://172.16.10.36:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:01 ctl01 kube-proxy[8230]: E1026 14:00:01.875286    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:01 ctl01 kubelet[8020]: E1026 14:00:01.932234    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:01 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:02 ctl01 kubelet[8020]: E1026 14:00:02.032550    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:02 ctl01 kubelet[8020]: E1026 14:00:02.100571    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://172.16.10.36:443/api/v1/pods?fieldSelector=spec.nodeName%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:02 ctl01 kubelet[8020]: E1026 14:00:02.101288    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://172.16.10.36:443/api/v1/nodes?fieldSelector=metadata.name%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:02 ctl01 kubelet[8020]: E1026 14:00:02.102311    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:02 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:02 ctl01 kubelet[8020]: E1026 14:00:02.132783    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:02 ctl01 kubelet[8020]: E1026 14:00:02.232965    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:02 ctl01 kubelet[8020]: E1026 14:00:02.333225    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:02 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:02 ctl01 kubelet[8020]: E1026 14:00:02.433463    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:02 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:02 ctl01 kubelet[8020]: E1026 14:00:02.533664    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:02 ctl01 kubelet[8020]: E1026 14:00:02.633910    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:02 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:02 ctl01 kubelet[8020]: E1026 14:00:02.734109    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:02 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:02 ctl01 kubelet[8020]: I1026 14:00:02.810156    8020 kubelet_node_status.go:279] Setting node annotation to enable volume controller attach/detach
Oct 26 14:00:02 ctl01 kubelet[8020]: I1026 14:00:02.810553    8020 setters.go:72] Using node IP: "172.16.10.36"
Oct 26 14:00:02 ctl01 kubelet[8020]: I1026 14:00:02.811870    8020 kubelet_node_status.go:447] Recording NodeHasSufficientMemory event message for node ctl01
Oct 26 14:00:02 ctl01 kubelet[8020]: I1026 14:00:02.811914    8020 kubelet_node_status.go:447] Recording NodeHasNoDiskPressure event message for node ctl01
Oct 26 14:00:02 ctl01 kubelet[8020]: I1026 14:00:02.811928    8020 kubelet_node_status.go:447] Recording NodeHasSufficientPID event message for node ctl01
Oct 26 14:00:02 ctl01 kubelet[8020]: I1026 14:00:02.811952    8020 kubelet_node_status.go:72] Attempting to register node ctl01
Oct 26 14:00:02 ctl01 kubelet[8020]: E1026 14:00:02.813928    8020 kubelet_node_status.go:94] Unable to register node "ctl01" with API server: Post https://172.16.10.36:443/api/v1/nodes: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:02 ctl01 kubelet[8020]: E1026 14:00:02.834324    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:02 ctl01 kube-proxy[8230]: E1026 14:00:02.836863    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: Get https://172.16.10.36:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:02 ctl01 kube-proxy[8230]: E1026 14:00:02.876098    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:02 ctl01 kubelet[8020]: E1026 14:00:02.934502    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:03 ctl01 kubelet[8020]: E1026 14:00:03.034705    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:03 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:03 ctl01 kubelet[8020]: E1026 14:00:03.101928    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://172.16.10.36:443/api/v1/pods?fieldSelector=spec.nodeName%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:03 ctl01 kubelet[8020]: E1026 14:00:03.102457    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://172.16.10.36:443/api/v1/nodes?fieldSelector=metadata.name%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:03 ctl01 kubelet[8020]: E1026 14:00:03.103665    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:03 ctl01 kubelet[8020]: E1026 14:00:03.134874    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:03 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:03 ctl01 kubelet[8020]: E1026 14:00:03.225079    8020 eviction_manager.go:247] eviction manager: failed to get summary stats: failed to get node info: node "ctl01" not found
Oct 26 14:00:03 ctl01 kubelet[8020]: E1026 14:00:03.235131    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:03 ctl01 kubelet[8020]: E1026 14:00:03.335398    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:03 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:03 ctl01 kubelet[8020]: E1026 14:00:03.435611    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:03 ctl01 kubelet[8020]: E1026 14:00:03.535792    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:03 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:03 ctl01 kubelet[8020]: E1026 14:00:03.636004    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:03 ctl01 kubelet[8020]: E1026 14:00:03.736330    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:03 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:03 ctl01 kubelet[8020]: E1026 14:00:03.836678    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:03 ctl01 kube-proxy[8230]: E1026 14:00:03.837997    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: Get https://172.16.10.36:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:03 ctl01 kube-proxy[8230]: E1026 14:00:03.876682    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:03 ctl01 kubelet[8020]: E1026 14:00:03.936908    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:03 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:04 ctl01 kubelet[8020]: E1026 14:00:04.037201    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:04 ctl01 kubelet[8020]: E1026 14:00:04.103116    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://172.16.10.36:443/api/v1/pods?fieldSelector=spec.nodeName%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:04 ctl01 kubelet[8020]: E1026 14:00:04.103880    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://172.16.10.36:443/api/v1/nodes?fieldSelector=metadata.name%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:04 ctl01 kubelet[8020]: E1026 14:00:04.105133    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:04 ctl01 kubelet[8020]: E1026 14:00:04.137533    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:04 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/control/init.sls'
Oct 26 14:00:04 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:04 ctl01 kubelet[8020]: E1026 14:00:04.237801    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:04 ctl01 kube-proxy[8230]: E1026 14:00:04.294788    8230 event.go:212] Unable to write event: 'Post https://172.16.10.36:443/api/v1/namespaces/default/events: dial tcp 172.16.10.36:443: connect: connection refused' (may retry after sleeping)
Oct 26 14:00:04 ctl01 kubelet[8020]: E1026 14:00:04.338169    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:04 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:04 ctl01 kubelet[8020]: E1026 14:00:04.438444    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:04 ctl01 kubelet[8020]: E1026 14:00:04.538641    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:04 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/control/service.sls'
Oct 26 14:00:04 ctl01 kubelet[8020]: E1026 14:00:04.638801    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:04 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:04 ctl01 kubelet[8020]: E1026 14:00:04.738981    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:04 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:04 ctl01 kube-proxy[8230]: E1026 14:00:04.839056    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: Get https://172.16.10.36:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:04 ctl01 kubelet[8020]: E1026 14:00:04.839200    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:04 ctl01 kube-proxy[8230]: E1026 14:00:04.877471    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:04 ctl01 kubelet[8020]: E1026 14:00:04.939368    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:04 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/control/role.sls'
Oct 26 14:00:05 ctl01 kubelet[8020]: E1026 14:00:05.039599    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:05 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:05 ctl01 kubelet[8020]: E1026 14:00:05.103826    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://172.16.10.36:443/api/v1/pods?fieldSelector=spec.nodeName%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:05 ctl01 kubelet[8020]: E1026 14:00:05.104725    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://172.16.10.36:443/api/v1/nodes?fieldSelector=metadata.name%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:05 ctl01 kubelet[8020]: E1026 14:00:05.106001    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:05 ctl01 kubelet[8020]: E1026 14:00:05.139764    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:05 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:05 ctl01 kubelet[8020]: E1026 14:00:05.240056    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:05 ctl01 kubelet[8020]: E1026 14:00:05.340331    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:05 ctl01 kubelet[8020]: E1026 14:00:05.440579    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:05 ctl01 kubelet[8020]: E1026 14:00:05.540819    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:05 ctl01 kubelet[8020]: E1026 14:00:05.641085    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:05 ctl01 kubelet[8020]: E1026 14:00:05.741346    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:05 ctl01 kube-proxy[8230]: E1026 14:00:05.840065    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: Get https://172.16.10.36:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:05 ctl01 kubelet[8020]: E1026 14:00:05.841614    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:05 ctl01 kube-proxy[8230]: E1026 14:00:05.878394    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:05 ctl01 kubelet[8020]: E1026 14:00:05.941831    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:06 ctl01 kubelet[8020]: E1026 14:00:06.042006    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:06 ctl01 kubelet[8020]: E1026 14:00:06.104466    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://172.16.10.36:443/api/v1/pods?fieldSelector=spec.nodeName%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:06 ctl01 kubelet[8020]: E1026 14:00:06.105311    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://172.16.10.36:443/api/v1/nodes?fieldSelector=metadata.name%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:06 ctl01 kubelet[8020]: E1026 14:00:06.106351    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:06 ctl01 kubelet[8020]: E1026 14:00:06.142140    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:06 ctl01 salt-minion[4526]: [INFO    ] Running state [curl] at time 14:00:06.231865
Oct 26 14:00:06 ctl01 salt-minion[4526]: [INFO    ] Executing state pkg.installed for [curl]
Oct 26 14:00:06 ctl01 salt-minion[4526]: [INFO    ] Executing command ['dpkg-query', '--showformat', '${Status} ${Package} ${Version} ${Architecture}', '-W'] in directory '/root'
Oct 26 14:00:06 ctl01 kubelet[8020]: E1026 14:00:06.242369    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:06 ctl01 kubelet[8020]: E1026 14:00:06.342609    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:06 ctl01 kubelet[8020]: E1026 14:00:06.442836    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:06 ctl01 kubelet[8020]: E1026 14:00:06.543033    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:06 ctl01 kubelet[8020]: E1026 14:00:06.643243    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:06 ctl01 salt-minion[4526]: [INFO    ] All specified packages are already installed
Oct 26 14:00:06 ctl01 salt-minion[4526]: [INFO    ] Completed state [curl] at time 14:00:06.659870 duration_in_ms=428.005
Oct 26 14:00:06 ctl01 salt-minion[4526]: [INFO    ] Running state [git] at time 14:00:06.660410
Oct 26 14:00:06 ctl01 salt-minion[4526]: [INFO    ] Executing state pkg.installed for [git]
Oct 26 14:00:06 ctl01 salt-minion[4526]: [INFO    ] All specified packages are already installed
Oct 26 14:00:06 ctl01 salt-minion[4526]: [INFO    ] Completed state [git] at time 14:00:06.670215 duration_in_ms=9.804
Oct 26 14:00:06 ctl01 salt-minion[4526]: [INFO    ] Running state [apt-transport-https] at time 14:00:06.670641
Oct 26 14:00:06 ctl01 salt-minion[4526]: [INFO    ] Executing state pkg.installed for [apt-transport-https]
Oct 26 14:00:06 ctl01 salt-minion[4526]: [INFO    ] All specified packages are already installed
Oct 26 14:00:06 ctl01 salt-minion[4526]: [INFO    ] Completed state [apt-transport-https] at time 14:00:06.680245 duration_in_ms=9.604
Oct 26 14:00:06 ctl01 salt-minion[4526]: [INFO    ] Running state [python-apt] at time 14:00:06.680670
Oct 26 14:00:06 ctl01 salt-minion[4526]: [INFO    ] Executing state pkg.installed for [python-apt]
Oct 26 14:00:06 ctl01 salt-minion[4526]: [INFO    ] All specified packages are already installed
Oct 26 14:00:06 ctl01 salt-minion[4526]: [INFO    ] Completed state [python-apt] at time 14:00:06.690148 duration_in_ms=9.478
Oct 26 14:00:06 ctl01 salt-minion[4526]: [INFO    ] Running state [socat] at time 14:00:06.690579
Oct 26 14:00:06 ctl01 salt-minion[4526]: [INFO    ] Executing state pkg.installed for [socat]
Oct 26 14:00:06 ctl01 salt-minion[4526]: [INFO    ] All specified packages are already installed
Oct 26 14:00:06 ctl01 salt-minion[4526]: [INFO    ] Completed state [socat] at time 14:00:06.700089 duration_in_ms=9.509
Oct 26 14:00:06 ctl01 salt-minion[4526]: [INFO    ] Running state [openssl] at time 14:00:06.700513
Oct 26 14:00:06 ctl01 salt-minion[4526]: [INFO    ] Executing state pkg.installed for [openssl]
Oct 26 14:00:06 ctl01 salt-minion[4526]: [INFO    ] All specified packages are already installed
Oct 26 14:00:06 ctl01 salt-minion[4526]: [INFO    ] Completed state [openssl] at time 14:00:06.710581 duration_in_ms=10.069
Oct 26 14:00:06 ctl01 salt-minion[4526]: [INFO    ] Running state [conntrack] at time 14:00:06.710997
Oct 26 14:00:06 ctl01 salt-minion[4526]: [INFO    ] Executing state pkg.installed for [conntrack]
Oct 26 14:00:06 ctl01 salt-minion[4526]: [INFO    ] All specified packages are already installed
Oct 26 14:00:06 ctl01 salt-minion[4526]: [INFO    ] Completed state [conntrack] at time 14:00:06.720384 duration_in_ms=9.387
Oct 26 14:00:06 ctl01 salt-minion[4526]: [INFO    ] Running state [/usr/bin/hyperkube] at time 14:00:06.723737
Oct 26 14:00:06 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/usr/bin/hyperkube]
Oct 26 14:00:06 ctl01 kubelet[8020]: E1026 14:00:06.743411    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:06 ctl01 kube-proxy[8230]: E1026 14:00:06.840825    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: Get https://172.16.10.36:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:06 ctl01 kubelet[8020]: E1026 14:00:06.843582    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:06 ctl01 kube-proxy[8230]: E1026 14:00:06.878909    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:06 ctl01 kubelet[8020]: E1026 14:00:06.943738    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:07 ctl01 kubelet[8020]: E1026 14:00:07.043927    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:07 ctl01 kubelet[8020]: E1026 14:00:07.105144    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://172.16.10.36:443/api/v1/pods?fieldSelector=spec.nodeName%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:07 ctl01 kubelet[8020]: E1026 14:00:07.105970    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://172.16.10.36:443/api/v1/nodes?fieldSelector=metadata.name%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:07 ctl01 kubelet[8020]: E1026 14:00:07.107081    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:07 ctl01 kubelet[8020]: E1026 14:00:07.144092    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:07 ctl01 kubelet[8020]: E1026 14:00:07.244322    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:07 ctl01 kubelet[8020]: E1026 14:00:07.344541    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:07 ctl01 kubelet[8020]: E1026 14:00:07.444785    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:07 ctl01 kubelet[8020]: E1026 14:00:07.545019    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:07 ctl01 kubelet[8020]: E1026 14:00:07.645185    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:07 ctl01 salt-minion[4526]: [INFO    ] File /usr/bin/hyperkube is in the correct state
Oct 26 14:00:07 ctl01 salt-minion[4526]: [INFO    ] Completed state [/usr/bin/hyperkube] at time 14:00:07.693281 duration_in_ms=969.543
Oct 26 14:00:07 ctl01 salt-minion[4526]: [INFO    ] Running state [/usr/bin/kubectl] at time 14:00:07.695648
Oct 26 14:00:07 ctl01 salt-minion[4526]: [INFO    ] Executing state file.symlink for [/usr/bin/kubectl]
Oct 26 14:00:07 ctl01 salt-minion[4526]: [INFO    ] Symlink /usr/bin/kubectl is present and owned by root:root
Oct 26 14:00:07 ctl01 salt-minion[4526]: [INFO    ] Completed state [/usr/bin/kubectl] at time 14:00:07.698966 duration_in_ms=3.318
Oct 26 14:00:07 ctl01 salt-minion[4526]: [INFO    ] Running state [containerd=1.2.5-3~u16.04+mcp] at time 14:00:07.699307
Oct 26 14:00:07 ctl01 salt-minion[4526]: [INFO    ] Executing state pkg.installed for [containerd=1.2.5-3~u16.04+mcp]
Oct 26 14:00:07 ctl01 salt-minion[4526]: [INFO    ] Executing command ['apt-cache', '-q', 'policy', 'containerd=1.2.5-3~u16.04+mcp'] in directory '/root'
Oct 26 14:00:07 ctl01 kubelet[8020]: E1026 14:00:07.745350    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:07 ctl01 kube-proxy[8230]: E1026 14:00:07.842077    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: Get https://172.16.10.36:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:07 ctl01 kubelet[8020]: E1026 14:00:07.845755    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:07 ctl01 salt-minion[4526]: [INFO    ] Executing command ['apt-get', '-q', 'update'] in directory '/root'
Oct 26 14:00:07 ctl01 kube-proxy[8230]: E1026 14:00:07.879821    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:07 ctl01 kubelet[8020]: E1026 14:00:07.946069    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:08 ctl01 kubelet[8020]: E1026 14:00:08.046360    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:08 ctl01 kubelet[8020]: E1026 14:00:08.106096    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://172.16.10.36:443/api/v1/pods?fieldSelector=spec.nodeName%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:08 ctl01 kubelet[8020]: E1026 14:00:08.108179    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:08 ctl01 kubelet[8020]: E1026 14:00:08.108175    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://172.16.10.36:443/api/v1/nodes?fieldSelector=metadata.name%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:08 ctl01 kubelet[8020]: E1026 14:00:08.146604    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:08 ctl01 kubelet[8020]: E1026 14:00:08.246813    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:08 ctl01 kubelet[8020]: E1026 14:00:08.250376    8020 event.go:212] Unable to write event: 'Post https://172.16.10.36:443/api/v1/namespaces/default/events: dial tcp 172.16.10.36:443: connect: connection refused' (may retry after sleeping)
Oct 26 14:00:08 ctl01 kubelet[8020]: E1026 14:00:08.347111    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:08 ctl01 kubelet[8020]: E1026 14:00:08.447334    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:08 ctl01 kubelet[8020]: E1026 14:00:08.547597    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:08 ctl01 kubelet[8020]: E1026 14:00:08.647897    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:08 ctl01 kubelet[8020]: E1026 14:00:08.748202    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:08 ctl01 kube-proxy[8230]: E1026 14:00:08.842807    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: Get https://172.16.10.36:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:08 ctl01 kubelet[8020]: E1026 14:00:08.848410    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:08 ctl01 kube-proxy[8230]: E1026 14:00:08.880652    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:08 ctl01 kubelet[8020]: E1026 14:00:08.948721    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:09 ctl01 kubelet[8020]: E1026 14:00:09.049107    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:09 ctl01 kubelet[8020]: E1026 14:00:09.106971    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://172.16.10.36:443/api/v1/pods?fieldSelector=spec.nodeName%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:09 ctl01 kubelet[8020]: E1026 14:00:09.109045    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:09 ctl01 kubelet[8020]: E1026 14:00:09.110051    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://172.16.10.36:443/api/v1/nodes?fieldSelector=metadata.name%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:09 ctl01 kubelet[8020]: E1026 14:00:09.149286    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:09 ctl01 kubelet[8020]: E1026 14:00:09.249575    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:09 ctl01 kubelet[8020]: E1026 14:00:09.349954    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:09 ctl01 kubelet[8020]: E1026 14:00:09.450246    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:09 ctl01 kubelet[8020]: E1026 14:00:09.550589    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:09 ctl01 kubelet[8020]: E1026 14:00:09.650859    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:09 ctl01 kubelet[8020]: E1026 14:00:09.751083    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:09 ctl01 kubelet[8020]: I1026 14:00:09.814274    8020 kubelet_node_status.go:279] Setting node annotation to enable volume controller attach/detach
Oct 26 14:00:09 ctl01 kubelet[8020]: I1026 14:00:09.814722    8020 setters.go:72] Using node IP: "172.16.10.36"
Oct 26 14:00:09 ctl01 kubelet[8020]: I1026 14:00:09.815860    8020 kubelet_node_status.go:447] Recording NodeHasSufficientMemory event message for node ctl01
Oct 26 14:00:09 ctl01 kubelet[8020]: I1026 14:00:09.815919    8020 kubelet_node_status.go:447] Recording NodeHasNoDiskPressure event message for node ctl01
Oct 26 14:00:09 ctl01 kubelet[8020]: I1026 14:00:09.815947    8020 kubelet_node_status.go:447] Recording NodeHasSufficientPID event message for node ctl01
Oct 26 14:00:09 ctl01 kubelet[8020]: I1026 14:00:09.815985    8020 kubelet_node_status.go:72] Attempting to register node ctl01
Oct 26 14:00:09 ctl01 kubelet[8020]: E1026 14:00:09.816529    8020 kubelet_node_status.go:94] Unable to register node "ctl01" with API server: Post https://172.16.10.36:443/api/v1/nodes: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:09 ctl01 kube-proxy[8230]: E1026 14:00:09.843563    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: Get https://172.16.10.36:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:09 ctl01 kubelet[8020]: E1026 14:00:09.851289    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:09 ctl01 kube-proxy[8230]: E1026 14:00:09.881587    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:09 ctl01 kubelet[8020]: E1026 14:00:09.951504    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:10 ctl01 kubelet[8020]: E1026 14:00:10.051729    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:10 ctl01 kubelet[8020]: E1026 14:00:10.107782    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://172.16.10.36:443/api/v1/pods?fieldSelector=spec.nodeName%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:10 ctl01 kubelet[8020]: E1026 14:00:10.109574    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:10 ctl01 kubelet[8020]: E1026 14:00:10.110578    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://172.16.10.36:443/api/v1/nodes?fieldSelector=metadata.name%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:10 ctl01 kubelet[8020]: E1026 14:00:10.151911    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:10 ctl01 kubelet[8020]: E1026 14:00:10.252132    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:10 ctl01 kubelet[8020]: E1026 14:00:10.352345    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:10 ctl01 salt-minion[4526]: [INFO    ] Executing command ['dpkg', '--get-selections', '*'] in directory '/root'
Oct 26 14:00:10 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemd-run', '--scope', 'apt-get', '-q', '-y', '-o', 'DPkg::Options::=--force-confold', '-o', 'DPkg::Options::=--force-confdef', 'install', 'containerd=1.2.5-3~u16.04+mcp'] in directory '/root'
Oct 26 14:00:10 ctl01 kubelet[8020]: E1026 14:00:10.452617    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:10 ctl01 systemd[1]: Started /usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold -o DPkg::Options::=--force-confdef install containerd=1.2.5-3~u16.04+mcp.
Oct 26 14:00:10 ctl01 kubelet[8020]: E1026 14:00:10.552865    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:10 ctl01 kubelet[8020]: E1026 14:00:10.653065    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:10 ctl01 kubelet[8020]: E1026 14:00:10.753254    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:10 ctl01 salt-minion[4526]: [INFO    ] Executing command ['dpkg-query', '--showformat', '${Status} ${Package} ${Version} ${Architecture}', '-W'] in directory '/root'
Oct 26 14:00:10 ctl01 salt-minion[4526]: [INFO    ] 1 targeted package was installed/updated.
Oct 26 14:00:10 ctl01 salt-minion[4526]: [INFO    ] Completed state [containerd=1.2.5-3~u16.04+mcp] at time 14:00:10.806906 duration_in_ms=3107.6
Oct 26 14:00:10 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/containerd/config.toml] at time 14:00:10.807234
Oct 26 14:00:10 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/containerd/config.toml]
Oct 26 14:00:10 ctl01 kube-proxy[8230]: E1026 14:00:10.844440    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: Get https://172.16.10.36:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:10 ctl01 kubelet[8020]: E1026 14:00:10.853461    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:10 ctl01 salt-minion[4526]: [INFO    ] User sudo_ubuntu Executing command saltutil.find_job with jid 20191026140010851232
Oct 26 14:00:10 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:10 ctl01 salt-minion[4526]: [INFO    ] Starting a new job with PID 9704
Oct 26 14:00:10 ctl01 kube-proxy[8230]: E1026 14:00:10.882332    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:10 ctl01 salt-minion[4526]: [INFO    ] Returning information for job: 20191026140010851232
Oct 26 14:00:10 ctl01 kubelet[8020]: E1026 14:00:10.953633    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:11 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:11 ctl01 kubelet[8020]: E1026 14:00:11.053817    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:11 ctl01 kubelet[8020]: E1026 14:00:11.108853    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://172.16.10.36:443/api/v1/pods?fieldSelector=spec.nodeName%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:11 ctl01 kubelet[8020]: E1026 14:00:11.110408    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:11 ctl01 kubelet[8020]: E1026 14:00:11.111453    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://172.16.10.36:443/api/v1/nodes?fieldSelector=metadata.name%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:11 ctl01 kubelet[8020]: E1026 14:00:11.154050    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:11 ctl01 salt-minion[4526]: [INFO    ] File /etc/containerd/config.toml is in the correct state
Oct 26 14:00:11 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/containerd/config.toml] at time 14:00:11.186100 duration_in_ms=378.865
Oct 26 14:00:11 ctl01 salt-minion[4526]: [INFO    ] Running state [containerd] at time 14:00:11.190331
Oct 26 14:00:11 ctl01 salt-minion[4526]: [INFO    ] Executing state service.running for [containerd]
Oct 26 14:00:11 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'status', 'containerd.service', '-n', '0'] in directory '/root'
Oct 26 14:00:11 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-active', 'containerd.service'] in directory '/root'
Oct 26 14:00:11 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-enabled', 'containerd.service'] in directory '/root'
Oct 26 14:00:11 ctl01 kubelet[8020]: E1026 14:00:11.254235    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:11 ctl01 salt-minion[4526]: [INFO    ] The service containerd is already running
Oct 26 14:00:11 ctl01 salt-minion[4526]: [INFO    ] Completed state [containerd] at time 14:00:11.255121 duration_in_ms=64.79
Oct 26 14:00:11 ctl01 salt-minion[4526]: [INFO    ] Running state [/tmp/crictl] at time 14:00:11.258322
Oct 26 14:00:11 ctl01 salt-minion[4526]: [INFO    ] Executing state archive.extracted for [/tmp/crictl]
Oct 26 14:00:11 ctl01 kubelet[8020]: E1026 14:00:11.354444    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:11 ctl01 kubelet[8020]: E1026 14:00:11.454606    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:11 ctl01 kubelet[8020]: E1026 14:00:11.554799    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:11 ctl01 salt-minion[4526]: [INFO    ] All files in archive are already present
Oct 26 14:00:11 ctl01 salt-minion[4526]: [INFO    ] Completed state [/tmp/crictl] at time 14:00:11.586927 duration_in_ms=328.605
Oct 26 14:00:11 ctl01 salt-minion[4526]: [INFO    ] Running state [/usr/local/bin/crictl] at time 14:00:11.588168
Oct 26 14:00:11 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/usr/local/bin/crictl]
Oct 26 14:00:11 ctl01 salt-minion[4526]: [WARNING ] Use of argument owner found, "owner" is invalid, please use "user"
Oct 26 14:00:11 ctl01 kubelet[8020]: E1026 14:00:11.655013    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:11 ctl01 kubelet[8020]: E1026 14:00:11.755243    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:11 ctl01 salt-minion[4526]: [INFO    ] File /usr/local/bin/crictl is in the correct state
Oct 26 14:00:11 ctl01 salt-minion[4526]: [INFO    ] Completed state [/usr/local/bin/crictl] at time 14:00:11.823028 duration_in_ms=234.86
Oct 26 14:00:11 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/crictl.yaml] at time 14:00:11.823321
Oct 26 14:00:11 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/crictl.yaml]
Oct 26 14:00:11 ctl01 kube-proxy[8230]: E1026 14:00:11.845547    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: Get https://172.16.10.36:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:11 ctl01 salt-minion[4526]: [INFO    ] File /etc/crictl.yaml is in the correct state
Oct 26 14:00:11 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/crictl.yaml] at time 14:00:11.847318 duration_in_ms=23.997
Oct 26 14:00:11 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/criproxy] at time 14:00:11.847682
Oct 26 14:00:11 ctl01 salt-minion[4526]: [INFO    ] Executing state file.absent for [/etc/criproxy]
Oct 26 14:00:11 ctl01 salt-minion[4526]: [INFO    ] File /etc/criproxy is not present
Oct 26 14:00:11 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/criproxy] at time 14:00:11.848893 duration_in_ms=1.21
Oct 26 14:00:11 ctl01 salt-minion[4526]: [INFO    ] Running state [criproxy] at time 14:00:11.849293
Oct 26 14:00:11 ctl01 salt-minion[4526]: [INFO    ] Executing state service.dead for [criproxy]
Oct 26 14:00:11 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'status', 'criproxy.service', '-n', '0'] in directory '/root'
Oct 26 14:00:11 ctl01 kubelet[8020]: E1026 14:00:11.855493    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:11 ctl01 salt-minion[4526]: [INFO    ] The named service criproxy is not available
Oct 26 14:00:11 ctl01 salt-minion[4526]: [INFO    ] Completed state [criproxy] at time 14:00:11.874034 duration_in_ms=24.74
Oct 26 14:00:11 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/systemd/system/kubelet.service] at time 14:00:11.874409
Oct 26 14:00:11 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/systemd/system/kubelet.service]
Oct 26 14:00:11 ctl01 kube-proxy[8230]: E1026 14:00:11.883103    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:11 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:11 ctl01 kubelet[8020]: E1026 14:00:11.955705    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:12 ctl01 kubelet[8020]: E1026 14:00:12.055884    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:12 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:12 ctl01 kubelet[8020]: E1026 14:00:12.109892    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://172.16.10.36:443/api/v1/pods?fieldSelector=spec.nodeName%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:12 ctl01 kubelet[8020]: E1026 14:00:12.111068    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:12 ctl01 kubelet[8020]: E1026 14:00:12.112258    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://172.16.10.36:443/api/v1/nodes?fieldSelector=metadata.name%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:12 ctl01 kubelet[8020]: E1026 14:00:12.156110    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:12 ctl01 salt-minion[4526]: [INFO    ] File /etc/systemd/system/kubelet.service is in the correct state
Oct 26 14:00:12 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/systemd/system/kubelet.service] at time 14:00:12.224399 duration_in_ms=349.989
Oct 26 14:00:12 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/config] at time 14:00:12.224908
Oct 26 14:00:12 ctl01 salt-minion[4526]: [INFO    ] Executing state file.absent for [/etc/kubernetes/config]
Oct 26 14:00:12 ctl01 salt-minion[4526]: [INFO    ] File /etc/kubernetes/config is not present
Oct 26 14:00:12 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/config] at time 14:00:12.226164 duration_in_ms=1.255
Oct 26 14:00:12 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/default/kubelet] at time 14:00:12.226531
Oct 26 14:00:12 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/default/kubelet]
Oct 26 14:00:12 ctl01 kubelet[8020]: E1026 14:00:12.256321    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:12 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:12 ctl01 kubelet[8020]: E1026 14:00:12.356508    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:12 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:12 ctl01 kubelet[8020]: E1026 14:00:12.456707    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:12 ctl01 kubelet[8020]: E1026 14:00:12.556898    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:12 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:12 ctl01 kubelet[8020]: E1026 14:00:12.657078    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:12 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:12 ctl01 kubelet[8020]: E1026 14:00:12.757275    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:12 ctl01 kube-proxy[8230]: E1026 14:00:12.846705    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: Get https://172.16.10.36:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:12 ctl01 kubelet[8020]: E1026 14:00:12.857471    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:12 ctl01 kube-proxy[8230]: E1026 14:00:12.883824    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:12 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:12 ctl01 kubelet[8020]: E1026 14:00:12.957672    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:13 ctl01 kubelet[8020]: E1026 14:00:13.058079    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:13 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:13 ctl01 kubelet[8020]: E1026 14:00:13.110731    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://172.16.10.36:443/api/v1/pods?fieldSelector=spec.nodeName%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:13 ctl01 kubelet[8020]: E1026 14:00:13.111534    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:13 ctl01 kubelet[8020]: E1026 14:00:13.112728    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://172.16.10.36:443/api/v1/nodes?fieldSelector=metadata.name%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:13 ctl01 kubelet[8020]: E1026 14:00:13.158270    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:13 ctl01 salt-minion[4526]: [INFO    ] File /etc/default/kubelet is in the correct state
Oct 26 14:00:13 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/default/kubelet] at time 14:00:13.215169 duration_in_ms=988.638
Oct 26 14:00:13 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/kubelet.kubeconfig] at time 14:00:13.215524
Oct 26 14:00:13 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/kubernetes/kubelet.kubeconfig]
Oct 26 14:00:13 ctl01 kubelet[8020]: E1026 14:00:13.225477    8020 eviction_manager.go:247] eviction manager: failed to get summary stats: failed to get node info: node "ctl01" not found
Oct 26 14:00:13 ctl01 kubelet[8020]: E1026 14:00:13.258471    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:13 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:13 ctl01 kubelet[8020]: E1026 14:00:13.358759    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:13 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:13 ctl01 kubelet[8020]: E1026 14:00:13.459367    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:13 ctl01 kubelet[8020]: E1026 14:00:13.559661    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:13 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:13 ctl01 kubelet[8020]: E1026 14:00:13.659955    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:13 ctl01 kubelet[8020]: E1026 14:00:13.760268    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:13 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:13 ctl01 kube-proxy[8230]: E1026 14:00:13.847465    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: Get https://172.16.10.36:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:13 ctl01 kubelet[8020]: E1026 14:00:13.860583    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:13 ctl01 kube-proxy[8230]: E1026 14:00:13.884393    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:13 ctl01 salt-minion[4526]: [INFO    ] File /etc/kubernetes/kubelet.kubeconfig is in the correct state
Oct 26 14:00:13 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/kubelet.kubeconfig] at time 14:00:13.921420 duration_in_ms=705.895
Oct 26 14:00:13 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/manifests] at time 14:00:13.921789
Oct 26 14:00:13 ctl01 salt-minion[4526]: [INFO    ] Executing state file.directory for [/etc/kubernetes/manifests]
Oct 26 14:00:13 ctl01 salt-minion[4526]: [INFO    ] Directory /etc/kubernetes/manifests is in the correct state
Oct 26 14:00:13 ctl01 salt-minion[4526]: Directory /etc/kubernetes/manifests updated
Oct 26 14:00:13 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/manifests] at time 14:00:13.923344 duration_in_ms=1.555
Oct 26 14:00:13 ctl01 salt-minion[4526]: [INFO    ] Running state [kubelet] at time 14:00:13.926407
Oct 26 14:00:13 ctl01 salt-minion[4526]: [INFO    ] Executing state service.running for [kubelet]
Oct 26 14:00:13 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'status', 'kubelet.service', '-n', '0'] in directory '/root'
Oct 26 14:00:13 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-active', 'kubelet.service'] in directory '/root'
Oct 26 14:00:13 ctl01 kubelet[8020]: E1026 14:00:13.960896    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:13 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-enabled', 'kubelet.service'] in directory '/root'
Oct 26 14:00:13 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemd-run', '--scope', 'systemctl', 'start', 'kubelet.service'] in directory '/root'
Oct 26 14:00:14 ctl01 systemd[1]: Started /bin/systemctl start kubelet.service.
Oct 26 14:00:14 ctl01 systemd[1]: Started Kubernetes Kubelet Server.
Oct 26 14:00:14 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-active', 'kubelet.service'] in directory '/root'
Oct 26 14:00:14 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-enabled', 'kubelet.service'] in directory '/root'
Oct 26 14:00:14 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-enabled', 'kubelet.service'] in directory '/root'
Oct 26 14:00:14 ctl01 kubelet[8020]: E1026 14:00:14.061059    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:14 ctl01 salt-minion[4526]: [INFO    ] {'kubelet': True}
Oct 26 14:00:14 ctl01 salt-minion[4526]: [INFO    ] Completed state [kubelet] at time 14:00:14.064718 duration_in_ms=138.311
Oct 26 14:00:14 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/logrotate.d/kubernetes] at time 14:00:14.065140
Oct 26 14:00:14 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/logrotate.d/kubernetes]
Oct 26 14:00:14 ctl01 salt-minion[4526]: [INFO    ] File /etc/logrotate.d/kubernetes is in the correct state
Oct 26 14:00:14 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/logrotate.d/kubernetes] at time 14:00:14.087888 duration_in_ms=22.747
Oct 26 14:00:14 ctl01 salt-minion[4526]: [INFO    ] Running state [/opt/cni/bin] at time 14:00:14.088319
Oct 26 14:00:14 ctl01 salt-minion[4526]: [INFO    ] Executing state archive.extracted for [/opt/cni/bin]
Oct 26 14:00:14 ctl01 kubelet[8020]: E1026 14:00:14.111360    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://172.16.10.36:443/api/v1/pods?fieldSelector=spec.nodeName%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:14 ctl01 kubelet[8020]: E1026 14:00:14.116509    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://172.16.10.36:443/api/v1/nodes?fieldSelector=metadata.name%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:14 ctl01 kubelet[8020]: E1026 14:00:14.116527    8020 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:14 ctl01 kubelet[9972]: Flag --pod-manifest-path has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 26 14:00:14 ctl01 kubelet[9972]: Flag --address has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 26 14:00:14 ctl01 kubelet[9972]: Flag --allow-privileged has been deprecated, will be removed in a future version
Oct 26 14:00:14 ctl01 kubelet[9972]: Flag --cluster-dns has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 26 14:00:14 ctl01 kubelet[9972]: Flag --cluster-domain has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 26 14:00:14 ctl01 kubelet[9972]: Flag --fail-swap-on has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.143906    9972 flags.go:33] FLAG: --address="172.16.10.36"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.143969    9972 flags.go:33] FLAG: --allow-privileged="true"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.143981    9972 flags.go:33] FLAG: --allowed-unsafe-sysctls="[]"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144001    9972 flags.go:33] FLAG: --alsologtostderr="false"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144013    9972 flags.go:33] FLAG: --anonymous-auth="true"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144021    9972 flags.go:33] FLAG: --application-metrics-count-limit="100"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144031    9972 flags.go:33] FLAG: --authentication-token-webhook="false"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144039    9972 flags.go:33] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144050    9972 flags.go:33] FLAG: --authorization-mode="AlwaysAllow"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144061    9972 flags.go:33] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144070    9972 flags.go:33] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144079    9972 flags.go:33] FLAG: --azure-container-registry-config=""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144087    9972 flags.go:33] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144097    9972 flags.go:33] FLAG: --bootstrap-checkpoint-path=""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144106    9972 flags.go:33] FLAG: --bootstrap-kubeconfig=""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144114    9972 flags.go:33] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144125    9972 flags.go:33] FLAG: --cgroup-driver="cgroupfs"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144134    9972 flags.go:33] FLAG: --cgroup-root=""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144142    9972 flags.go:33] FLAG: --cgroups-per-qos="true"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144150    9972 flags.go:33] FLAG: --chaos-chance="0"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144162    9972 flags.go:33] FLAG: --client-ca-file=""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144170    9972 flags.go:33] FLAG: --cloud-config=""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144178    9972 flags.go:33] FLAG: --cloud-provider=""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144186    9972 flags.go:33] FLAG: --cluster-dns="[10.254.0.10]"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144201    9972 flags.go:33] FLAG: --cluster-domain="cluster.local"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144209    9972 flags.go:33] FLAG: --cni-bin-dir="/opt/cni/bin"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144218    9972 flags.go:33] FLAG: --cni-conf-dir="/etc/cni/net.d"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144227    9972 flags.go:33] FLAG: --config=""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144236    9972 flags.go:33] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144245    9972 flags.go:33] FLAG: --container-log-max-files="5"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144257    9972 flags.go:33] FLAG: --container-log-max-size="10Mi"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144265    9972 flags.go:33] FLAG: --container-runtime="remote"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144281    9972 flags.go:33] FLAG: --container-runtime-endpoint="unix:///run/containerd/containerd.sock"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144295    9972 flags.go:33] FLAG: --containerd="unix:///var/run/containerd.sock"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144305    9972 flags.go:33] FLAG: --containerized="false"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144313    9972 flags.go:33] FLAG: --contention-profiling="false"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144322    9972 flags.go:33] FLAG: --cpu-cfs-quota="true"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144330    9972 flags.go:33] FLAG: --cpu-cfs-quota-period="100ms"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144339    9972 flags.go:33] FLAG: --cpu-manager-policy="none"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144348    9972 flags.go:33] FLAG: --cpu-manager-reconcile-period="10s"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144356    9972 flags.go:33] FLAG: --docker="unix:///var/run/docker.sock"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144366    9972 flags.go:33] FLAG: --docker-endpoint="unix:///var/run/docker.sock"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144375    9972 flags.go:33] FLAG: --docker-env-metadata-whitelist=""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144383    9972 flags.go:33] FLAG: --docker-only="false"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144391    9972 flags.go:33] FLAG: --docker-root="/var/lib/docker"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144400    9972 flags.go:33] FLAG: --docker-tls="false"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144408    9972 flags.go:33] FLAG: --docker-tls-ca="ca.pem"
Oct 26 14:00:14 ctl01 systemd[1]: Started Kubernetes systemd probe.
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144417    9972 flags.go:33] FLAG: --docker-tls-cert="cert.pem"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144425    9972 flags.go:33] FLAG: --docker-tls-key="key.pem"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144434    9972 flags.go:33] FLAG: --dynamic-config-dir=""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144446    9972 flags.go:33] FLAG: --enable-controller-attach-detach="true"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144455    9972 flags.go:33] FLAG: --enable-debugging-handlers="true"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144463    9972 flags.go:33] FLAG: --enable-load-reader="false"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144471    9972 flags.go:33] FLAG: --enable-server="true"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144480    9972 flags.go:33] FLAG: --enforce-node-allocatable="[pods]"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144496    9972 flags.go:33] FLAG: --event-burst="10"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144505    9972 flags.go:33] FLAG: --event-qps="5"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144513    9972 flags.go:33] FLAG: --event-storage-age-limit="default=0"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144522    9972 flags.go:33] FLAG: --event-storage-event-limit="default=0"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144531    9972 flags.go:33] FLAG: --eviction-hard="imagefs.available<15%,memory.available<100Mi,nodefs.available<10%,nodefs.inodesFree<5%"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144558    9972 flags.go:33] FLAG: --eviction-max-pod-grace-period="0"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144567    9972 flags.go:33] FLAG: --eviction-minimum-reclaim=""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144580    9972 flags.go:33] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144588    9972 flags.go:33] FLAG: --eviction-soft=""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144603    9972 flags.go:33] FLAG: --eviction-soft-grace-period=""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144616    9972 flags.go:33] FLAG: --exit-on-lock-contention="false"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144625    9972 flags.go:33] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144633    9972 flags.go:33] FLAG: --experimental-bootstrap-kubeconfig=""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144642    9972 flags.go:33] FLAG: --experimental-check-node-capabilities-before-mount="false"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144651    9972 flags.go:33] FLAG: --experimental-dockershim="false"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144659    9972 flags.go:33] FLAG: --experimental-dockershim-root-directory="/var/lib/dockershim"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144668    9972 flags.go:33] FLAG: --experimental-fail-swap-on="true"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144676    9972 flags.go:33] FLAG: --experimental-kernel-memcg-notification="false"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144684    9972 flags.go:33] FLAG: --experimental-mounter-path=""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144692    9972 flags.go:33] FLAG: --fail-swap-on="true"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144701    9972 flags.go:33] FLAG: --feature-gates=""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144712    9972 flags.go:33] FLAG: --file-check-frequency="20s"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144721    9972 flags.go:33] FLAG: --global-housekeeping-interval="1m0s"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144730    9972 flags.go:33] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144739    9972 flags.go:33] FLAG: --healthz-bind-address="127.0.0.1"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144748    9972 flags.go:33] FLAG: --healthz-port="10248"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144758    9972 flags.go:33] FLAG: --help="false"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144767    9972 flags.go:33] FLAG: --host-ipc-sources="[*]"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144782    9972 flags.go:33] FLAG: --host-network-sources="[*]"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144798    9972 flags.go:33] FLAG: --host-pid-sources="[*]"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144811    9972 flags.go:33] FLAG: --hostname-override="ctl01"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144819    9972 flags.go:33] FLAG: --housekeeping-interval="10s"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144828    9972 flags.go:33] FLAG: --http-check-frequency="20s"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144836    9972 flags.go:33] FLAG: --image-gc-high-threshold="85"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144845    9972 flags.go:33] FLAG: --image-gc-low-threshold="80"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144853    9972 flags.go:33] FLAG: --image-pull-progress-deadline="1m0s"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144861    9972 flags.go:33] FLAG: --image-service-endpoint=""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144869    9972 flags.go:33] FLAG: --iptables-drop-bit="15"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144877    9972 flags.go:33] FLAG: --iptables-masquerade-bit="14"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144885    9972 flags.go:33] FLAG: --keep-terminated-pod-volumes="false"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144894    9972 flags.go:33] FLAG: --kube-api-burst="10"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144908    9972 flags.go:33] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144922    9972 flags.go:33] FLAG: --kube-api-qps="5"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144931    9972 flags.go:33] FLAG: --kube-reserved=""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144941    9972 flags.go:33] FLAG: --kube-reserved-cgroup=""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144949    9972 flags.go:33] FLAG: --kubeconfig="/etc/kubernetes/kubelet.kubeconfig"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144958    9972 flags.go:33] FLAG: --kubelet-cgroups=""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144966    9972 flags.go:33] FLAG: --lock-file=""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.144975    9972 flags.go:33] FLAG: --log-backtrace-at=":0"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145153    9972 flags.go:33] FLAG: --log-cadvisor-usage="false"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145167    9972 flags.go:33] FLAG: --log-dir=""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145173    9972 flags.go:33] FLAG: --log-file=""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145179    9972 flags.go:33] FLAG: --log-flush-frequency="5s"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145187    9972 flags.go:33] FLAG: --logtostderr="true"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145193    9972 flags.go:33] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145200    9972 flags.go:33] FLAG: --make-iptables-util-chains="true"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145207    9972 flags.go:33] FLAG: --manifest-url=""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145213    9972 flags.go:33] FLAG: --manifest-url-header=""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145225    9972 flags.go:33] FLAG: --master-service-namespace="default"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145232    9972 flags.go:33] FLAG: --max-open-files="1000000"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145241    9972 flags.go:33] FLAG: --max-pods="110"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145248    9972 flags.go:33] FLAG: --maximum-dead-containers="-1"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145254    9972 flags.go:33] FLAG: --maximum-dead-containers-per-container="1"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145260    9972 flags.go:33] FLAG: --minimum-container-ttl-duration="0s"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145265    9972 flags.go:33] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145271    9972 flags.go:33] FLAG: --network-plugin="cni"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145277    9972 flags.go:33] FLAG: --network-plugin-mtu="0"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145283    9972 flags.go:33] FLAG: --node-ip="172.16.10.36"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145289    9972 flags.go:33] FLAG: --node-labels="node-role.kubernetes.io/master=true"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145302    9972 flags.go:33] FLAG: --node-status-max-images="50"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145308    9972 flags.go:33] FLAG: --node-status-update-frequency="10s"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145314    9972 flags.go:33] FLAG: --non-masquerade-cidr="10.0.0.0/8"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145319    9972 flags.go:33] FLAG: --oom-score-adj="-999"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145325    9972 flags.go:33] FLAG: --pod-cidr=""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145331    9972 flags.go:33] FLAG: --pod-infra-container-image="docker-prod-local.artifactory.mirantis.com/mirantis/kubernetes/pause-amd64:v1.13.5-3"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145341    9972 flags.go:33] FLAG: --pod-manifest-path="/etc/kubernetes/manifests"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145347    9972 flags.go:33] FLAG: --pod-max-pids="-1"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145365    9972 flags.go:33] FLAG: --pods-per-core="0"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145371    9972 flags.go:33] FLAG: --port="10250"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145377    9972 flags.go:33] FLAG: --protect-kernel-defaults="false"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145382    9972 flags.go:33] FLAG: --provider-id=""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145400    9972 flags.go:33] FLAG: --qos-reserved=""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145408    9972 flags.go:33] FLAG: --read-only-port="10255"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145414    9972 flags.go:33] FLAG: --really-crash-for-testing="false"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145419    9972 flags.go:33] FLAG: --redirect-container-streaming="false"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145425    9972 flags.go:33] FLAG: --register-node="true"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145431    9972 flags.go:33] FLAG: --register-schedulable="true"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145436    9972 flags.go:33] FLAG: --register-with-taints=""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145444    9972 flags.go:33] FLAG: --registry-burst="10"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145449    9972 flags.go:33] FLAG: --registry-qps="5"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145455    9972 flags.go:33] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 26 14:00:14 ctl01 kubelet[8020]: E1026 14:00:14.166788    8020 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:14 ctl01 kubelet[8020]: W1026 14:00:14.167271    8020 raw.go:87] Error while processing event ("/sys/fs/cgroup/cpu,cpuacct/system.slice/run-r52846df39ae841ec986121c80a6aa36a.scope": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/cpu,cpuacct/system.slice/run-r52846df39ae841ec986121c80a6aa36a.scope: no such file or directory
Oct 26 14:00:14 ctl01 kubelet[8020]: W1026 14:00:14.167327    8020 raw.go:87] Error while processing event ("/sys/fs/cgroup/blkio/system.slice/run-r52846df39ae841ec986121c80a6aa36a.scope": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/blkio/system.slice/run-r52846df39ae841ec986121c80a6aa36a.scope: no such file or directory
Oct 26 14:00:14 ctl01 kubelet[8020]: W1026 14:00:14.167363    8020 raw.go:87] Error while processing event ("/sys/fs/cgroup/memory/system.slice/run-r52846df39ae841ec986121c80a6aa36a.scope": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/memory/system.slice/run-r52846df39ae841ec986121c80a6aa36a.scope: no such file or directory
Oct 26 14:00:14 ctl01 kubelet[8020]: W1026 14:00:14.167391    8020 raw.go:87] Error while processing event ("/sys/fs/cgroup/devices/system.slice/run-r52846df39ae841ec986121c80a6aa36a.scope": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/devices/system.slice/run-r52846df39ae841ec986121c80a6aa36a.scope: no such file or directory
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145461    9972 flags.go:33] FLAG: --root-dir="/var/lib/kubelet"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145467    9972 flags.go:33] FLAG: --rotate-certificates="false"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145472    9972 flags.go:33] FLAG: --rotate-server-certificates="false"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145477    9972 flags.go:33] FLAG: --runonce="false"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145483    9972 flags.go:33] FLAG: --runtime-cgroups=""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145488    9972 flags.go:33] FLAG: --runtime-request-timeout="2m0s"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145494    9972 flags.go:33] FLAG: --seccomp-profile-root="/var/lib/kubelet/seccomp"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145500    9972 flags.go:33] FLAG: --serialize-image-pulls="true"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145505    9972 flags.go:33] FLAG: --stderrthreshold="2"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145511    9972 flags.go:33] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145518    9972 flags.go:33] FLAG: --storage-driver-db="cadvisor"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145524    9972 flags.go:33] FLAG: --storage-driver-host="localhost:8086"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145530    9972 flags.go:33] FLAG: --storage-driver-password="root"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145535    9972 flags.go:33] FLAG: --storage-driver-secure="false"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145541    9972 flags.go:33] FLAG: --storage-driver-table="stats"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145546    9972 flags.go:33] FLAG: --storage-driver-user="root"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145552    9972 flags.go:33] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145558    9972 flags.go:33] FLAG: --sync-frequency="1m0s"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145563    9972 flags.go:33] FLAG: --system-cgroups=""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145569    9972 flags.go:33] FLAG: --system-reserved=""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145575    9972 flags.go:33] FLAG: --system-reserved-cgroup=""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145580    9972 flags.go:33] FLAG: --tls-cert-file=""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145595    9972 flags.go:33] FLAG: --tls-cipher-suites="[]"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145720    9972 flags.go:33] FLAG: --tls-min-version=""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145726    9972 flags.go:33] FLAG: --tls-private-key-file=""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145731    9972 flags.go:33] FLAG: --v="2"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145737    9972 flags.go:33] FLAG: --version="false"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145747    9972 flags.go:33] FLAG: --vmodule=""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145753    9972 flags.go:33] FLAG: --volume-plugin-dir="/usr/libexec/kubernetes/kubelet-plugins/volume/exec/"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145760    9972 flags.go:33] FLAG: --volume-stats-agg-period="1m0s"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145794    9972 feature_gate.go:206] feature gates: &{map[]}
Oct 26 14:00:14 ctl01 kubelet[9972]: W1026 14:00:14.145815    9972 options.go:265] unknown 'kubernetes.io' or 'k8s.io' labels specified with --node-labels: [node-role.kubernetes.io/master]
Oct 26 14:00:14 ctl01 kubelet[9972]: W1026 14:00:14.145828    9972 options.go:266] in 1.15, --node-labels in the 'kubernetes.io' namespace must begin with an allowed prefix (kubelet.kubernetes.io, node.kubernetes.io) or be in the specifically allowed set (beta.kubernetes.io/arch, beta.kubernetes.io/instance-type, beta.kubernetes.io/os, failure-domain.beta.kubernetes.io/region, failure-domain.beta.kubernetes.io/zone, failure-domain.kubernetes.io/region, failure-domain.kubernetes.io/zone, kubernetes.io/arch, kubernetes.io/hostname, kubernetes.io/instance-type, kubernetes.io/os)
Oct 26 14:00:14 ctl01 kubelet[9972]: W1026 14:00:14.145841    9972 server.go:182] Warning: For remote container runtime, --pod-infra-container-image is ignored in kubelet, which should be set in that remote runtime instead
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.145884    9972 feature_gate.go:206] feature gates: &{map[]}
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.151084    9972 mount_linux.go:180] Detected OS with systemd
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.151126    9972 server.go:407] Version: v1.13.5-3+98374c02d2d8c1
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.151163    9972 feature_gate.go:206] feature gates: &{map[]}
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.151212    9972 feature_gate.go:206] feature gates: &{map[]}
Oct 26 14:00:14 ctl01 kubelet[9972]: W1026 14:00:14.151230    9972 options.go:265] unknown 'kubernetes.io' or 'k8s.io' labels specified with --node-labels: [node-role.kubernetes.io/master]
Oct 26 14:00:14 ctl01 systemd[1]: kubelet.service: Main process exited, code=exited, status=255/n/a
Oct 26 14:00:14 ctl01 kubelet[9972]: W1026 14:00:14.151242    9972 options.go:266] in 1.15, --node-labels in the 'kubernetes.io' namespace must begin with an allowed prefix (kubelet.kubernetes.io, node.kubernetes.io) or be in the specifically allowed set (beta.kubernetes.io/arch, beta.kubernetes.io/instance-type, beta.kubernetes.io/os, failure-domain.beta.kubernetes.io/region, failure-domain.beta.kubernetes.io/zone, failure-domain.kubernetes.io/region, failure-domain.kubernetes.io/zone, kubernetes.io/arch, kubernetes.io/hostname, kubernetes.io/instance-type, kubernetes.io/os)
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.151311    9972 plugins.go:103] No cloud provider specified.
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.151326    9972 server.go:523] No cloud provider specified: "" from the config file: ""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.159401    9972 manager.go:155] cAdvisor running in container: "/sys/fs/cgroup/cpu,cpuacct/system.slice/kubelet.service"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.160411    9972 fs.go:142] Filesystem UUIDs: map[2019-10-26-15-49-44-00:/dev/sr0 3dc7f99a-1e06-428f-86bf-f1f8932b4515:/dev/vda1]
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.160428    9972 fs.go:143] Filesystem partitions: map[tmpfs:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /dev/vda1:{mountpoint:/ major:252 minor:1 fsType:ext4 blockSize:0}]
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.165162    9972 manager.go:229] Machine: {NumCores:8 CpuFrequency:2799994 MemoryCapacity:14705016832 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:70a4fc47e8254bb0a9da2f2beeb1a9ad SystemUUID:B60BACF9-7C5C-402C-99A3-F64133E62AD6 BootID:f6b0e736-be69-43fe-8cfa-5f08c44a9aa3 Filesystems:[{Device:/dev/vda1 DeviceMajor:252 DeviceMinor:1 Capacity:104022159360 Type:vfs Inodes:12800000 HasInodes:true} {Device:tmpfs DeviceMajor:0 DeviceMinor:24 Capacity:1470504960 Type:vfs Inodes:1795046 HasInodes:true}] DiskMap:map[252:0:{Name:vda Major:252 Minor:0 Size:107374182400 Scheduler:none}] NetworkDevices:[{Name:br-mgmt MacAddress:52:54:00:cc:bf:d7 Speed:0 Mtu:9000} {Name:ens3 MacAddress:52:54:00:d1:44:f2 Speed:-1 Mtu:9000} {Name:ens4 MacAddress:52:54:00:cc:bf:d7 Speed:-1 Mtu:9000} {Name:ens5 MacAddress:52:54:00:dd:4a:72 Speed:-1 Mtu:9000} {Name:ens5.1000 MacAddress:52:54:00:dd:4a:72 Speed:-1 Mtu:9000} {Name:ens6 MacAddress:52:54:00:23:fb:5b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:14705016832 Cores:[{Id:0 Threads:[0] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]} {Id:1 Memory:0 Cores:[{Id:0 Threads:[1] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]} {Id:2 Memory:0 Cores:[{Id:0 Threads:[2] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]} {Id:3 Memory:0 Cores:[{Id:0 Threads:[3] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]} {Id:4 Memory:0 Cores:[{Id:0 Threads:[4] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]} {Id:5 Memory:0 Cores:[{Id:0 Threads:[5] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]} {Id:6 Memory:0 Cores:
Oct 26 14:00:14 ctl01 kubelet[9972]: [{Id:0 Threads:[6] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]} {Id:7 Memory:0 Cores:[{Id:0 Threads:[7] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.165505    9972 manager.go:235] Version: {KernelVersion:4.15.0-66-generic ContainerOsVersion:Ubuntu 16.04.6 LTS DockerVersion:Unknown DockerAPIVersion:Unknown CadvisorVersion: CadvisorRevision:}
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.165613    9972 server.go:666] --cgroups-per-qos enabled, but --cgroup-root was not specified.  defaulting to /
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.165872    9972 container_manager_linux.go:248] container manager verified user specified cgroup-root exists: []
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.165887    9972 container_manager_linux.go:253] Creating Container Manager object based on Node Config: {RuntimeCgroupsName: SystemCgroupsName: KubeletCgroupsName: ContainerRuntime:remote CgroupsPerQOS:true CgroupRoot:/ CgroupDriver:cgroupfs KubeletRootDir:/var/lib/kubelet ProtectKernelDefaults:false NodeAllocatableConfig:{KubeReservedCgroupName: SystemReservedCgroupName: EnforceNodeAllocatable:map[pods:{}] KubeReserved:map[] SystemReserved:map[] HardEvictionThresholds:[{Signal:memory.available Operator:LessThan Value:{Quantity:100Mi Percentage:0} GracePeriod:0s MinReclaim:<nil>} {Signal:nodefs.available Operator:LessThan Value:{Quantity:<nil> Percentage:0.1} GracePeriod:0s MinReclaim:<nil>} {Signal:nodefs.inodesFree Operator:LessThan Value:{Quantity:<nil> Percentage:0.05} GracePeriod:0s MinReclaim:<nil>} {Signal:imagefs.available Operator:LessThan Value:{Quantity:<nil> Percentage:0.15} GracePeriod:0s MinReclaim:<nil>}]} QOSReserved:map[] ExperimentalCPUManagerPolicy:none ExperimentalCPUManagerReconcilePeriod:10s ExperimentalPodPidsLimit:-1 EnforceCPULimits:true CPUCFSQuotaPeriod:100ms}
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.165982    9972 container_manager_linux.go:272] Creating device plugin manager: true
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.165992    9972 manager.go:109] Creating Device Plugin manager at /var/lib/kubelet/device-plugins/kubelet.sock
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.166019    9972 state_mem.go:36] [cpumanager] initializing new in-memory state store
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.166122    9972 state_mem.go:84] [cpumanager] updated default cpuset: ""
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.166135    9972 state_mem.go:92] [cpumanager] updated cpuset assignments: "map[]"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.166144    9972 state_checkpoint.go:100] [cpumanager] state checkpoint: restored state from checkpoint
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.166151    9972 state_checkpoint.go:101] [cpumanager] state checkpoint: defaultCPUSet:
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.166240    9972 server.go:941] Using root directory: /var/lib/kubelet
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.166264    9972 kubelet.go:281] Adding pod path: /etc/kubernetes/manifests
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.166295    9972 file.go:68] Watching path "/etc/kubernetes/manifests"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.166310    9972 kubelet.go:306] Watching apiserver
Oct 26 14:00:14 ctl01 kubelet[9972]: E1026 14:00:14.166839    9972 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://172.16.10.36:443/api/v1/pods?fieldSelector=spec.nodeName%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:14 ctl01 kubelet[9972]: E1026 14:00:14.167194    9972 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:14 ctl01 kubelet[9972]: E1026 14:00:14.167250    9972 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://172.16.10.36:443/api/v1/nodes?fieldSelector=metadata.name%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.170553    9972 kuberuntime_manager.go:198] Container runtime containerd initialized, version: 1.2.5-3, apiVersion: v1alpha2
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.170796    9972 plugins.go:547] Loaded volume plugin "kubernetes.io/aws-ebs"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.170812    9972 plugins.go:547] Loaded volume plugin "kubernetes.io/empty-dir"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.170822    9972 plugins.go:547] Loaded volume plugin "kubernetes.io/gce-pd"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.170832    9972 plugins.go:547] Loaded volume plugin "kubernetes.io/git-repo"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.170841    9972 plugins.go:547] Loaded volume plugin "kubernetes.io/host-path"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.170851    9972 plugins.go:547] Loaded volume plugin "kubernetes.io/nfs"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.170860    9972 plugins.go:547] Loaded volume plugin "kubernetes.io/secret"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.170870    9972 plugins.go:547] Loaded volume plugin "kubernetes.io/iscsi"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.170881    9972 plugins.go:547] Loaded volume plugin "kubernetes.io/glusterfs"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.170899    9972 plugins.go:547] Loaded volume plugin "kubernetes.io/rbd"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.170909    9972 plugins.go:547] Loaded volume plugin "kubernetes.io/cinder"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.170919    9972 plugins.go:547] Loaded volume plugin "kubernetes.io/quobyte"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.170928    9972 plugins.go:547] Loaded volume plugin "kubernetes.io/cephfs"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.170947    9972 plugins.go:547] Loaded volume plugin "kubernetes.io/downward-api"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.170957    9972 plugins.go:547] Loaded volume plugin "kubernetes.io/fc"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.170966    9972 plugins.go:547] Loaded volume plugin "kubernetes.io/flocker"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.170975    9972 plugins.go:547] Loaded volume plugin "kubernetes.io/azure-file"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.170985    9972 plugins.go:547] Loaded volume plugin "kubernetes.io/configmap"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.171000    9972 plugins.go:547] Loaded volume plugin "kubernetes.io/vsphere-volume"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.171009    9972 plugins.go:547] Loaded volume plugin "kubernetes.io/azure-disk"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.171018    9972 plugins.go:547] Loaded volume plugin "kubernetes.io/photon-pd"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.171028    9972 plugins.go:547] Loaded volume plugin "kubernetes.io/projected"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.171047    9972 plugins.go:547] Loaded volume plugin "kubernetes.io/portworx-volume"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.171057    9972 plugins.go:547] Loaded volume plugin "kubernetes.io/scaleio"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.171067    9972 plugins.go:547] Loaded volume plugin "kubernetes.io/local-volume"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.171076    9972 plugins.go:547] Loaded volume plugin "kubernetes.io/storageos"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.171090    9972 plugins.go:547] Loaded volume plugin "kubernetes.io/csi"
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.171342    9972 server.go:999] Started kubelet
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.171735    9972 server.go:137] Starting to listen on 172.16.10.36:10250
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.171760    9972 server.go:157] Starting to listen read-only on 172.16.10.36:10255
Oct 26 14:00:14 ctl01 kubelet[9972]: E1026 14:00:14.172108    9972 event.go:212] Unable to write event: 'Post https://172.16.10.36:443/api/v1/namespaces/default/events: dial tcp 172.16.10.36:443: connect: connection refused' (may retry after sleeping)
Oct 26 14:00:14 ctl01 kubelet[9972]: E1026 14:00:14.172165    9972 server.go:752] Starting health server failed: listen tcp 127.0.0.1:10248: bind: address already in use
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.172705    9972 fs_resource_analyzer.go:66] Starting FS ResourceAnalyzer
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.172742    9972 server.go:333] Adding debug handlers to kubelet server.
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.172743    9972 status_manager.go:152] Starting to sync pod status with apiserver
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.172775    9972 kubelet.go:1829] Starting kubelet main sync loop.
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.172797    9972 kubelet.go:1846] skipping pod synchronization - [container runtime status check may not have completed yet PLEG is not healthy: pleg has yet to be successful]
Oct 26 14:00:14 ctl01 kubelet[9972]: E1026 14:00:14.173946    9972 cri_stats_provider.go:320] Failed to get the info of the filesystem with mountpoint "/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs": unable to find data in memory cache.
Oct 26 14:00:14 ctl01 kubelet[9972]: E1026 14:00:14.174035    9972 kubelet.go:1308] Image garbage collection failed once. Stats initialization may not have completed yet: invalid capacity 0 on image filesystem
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.174572    9972 volume_manager.go:246] The desired_state_of_world populator starts
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.174581    9972 volume_manager.go:248] Starting Kubelet Volume Manager
Oct 26 14:00:14 ctl01 kubelet[9972]: I1026 14:00:14.174663    9972 desired_state_of_world_populator.go:130] Desired state populator starts to run
Oct 26 14:00:14 ctl01 kubelet[9972]: F1026 14:00:14.174945    9972 server.go:149] listen tcp 172.16.10.36:10250: bind: address already in use
Oct 26 14:00:14 ctl01 systemd[1]: kubelet.service: Unit entered failed state.
Oct 26 14:00:14 ctl01 systemd[1]: kubelet.service: Failed with result 'exit-code'.
Oct 26 14:00:14 ctl01 kube-proxy[8230]: E1026 14:00:14.295690    8230 event.go:212] Unable to write event: 'Post https://172.16.10.36:443/api/v1/namespaces/default/events: dial tcp 172.16.10.36:443: connect: connection refused' (may retry after sleeping)
Oct 26 14:00:14 ctl01 kube-proxy[8230]: E1026 14:00:14.848403    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: Get https://172.16.10.36:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:14 ctl01 kube-proxy[8230]: E1026 14:00:14.885295    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:15 ctl01 salt-minion[4526]: [INFO    ] Executing command ['tar', 'xz', '-f', '/var/cache/salt/minion/extrn_files/base/docker-prod-local.artifactory.mirantis.com/artifactory/binary-prod-local/mirantis/kubernetes/containernetworking-plugins/containernetworking-plugins_v0.7.2-173-g8db2808.tar.gz'] in directory '/opt/cni/bin/'
Oct 26 14:00:15 ctl01 salt-minion[4526]: [INFO    ] Executing command ['tar', '--version'] in directory '/root'
Oct 26 14:00:15 ctl01 salt-minion[4526]: [INFO    ] {'extracted_files': 'no tar output so far'}
Oct 26 14:00:15 ctl01 salt-minion[4526]: [INFO    ] Completed state [/opt/cni/bin] at time 14:00:15.730935 duration_in_ms=1642.615
Oct 26 14:00:15 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/addons] at time 14:00:15.731670
Oct 26 14:00:15 ctl01 salt-minion[4526]: [INFO    ] Executing state file.directory for [/etc/kubernetes/addons]
Oct 26 14:00:15 ctl01 salt-minion[4526]: [INFO    ] Directory /etc/kubernetes/addons is in the correct state
Oct 26 14:00:15 ctl01 salt-minion[4526]: Directory /etc/kubernetes/addons updated
Oct 26 14:00:15 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/addons] at time 14:00:15.734719 duration_in_ms=3.047
Oct 26 14:00:15 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/addons/calico/calico-kube-controllers.yml] at time 14:00:15.735442
Oct 26 14:00:15 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/kubernetes/addons/calico/calico-kube-controllers.yml]
Oct 26 14:00:15 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:15 ctl01 kube-proxy[8230]: E1026 14:00:15.849167    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: Get https://172.16.10.36:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:15 ctl01 kube-proxy[8230]: E1026 14:00:15.885858    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:15 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:16 ctl01 salt-minion[4526]: [INFO    ] File /etc/kubernetes/addons/calico/calico-kube-controllers.yml is in the correct state
Oct 26 14:00:16 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/addons/calico/calico-kube-controllers.yml] at time 14:00:16.092322 duration_in_ms=356.88
Oct 26 14:00:16 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/addons/calico/calico-rbac.yml] at time 14:00:16.093022
Oct 26 14:00:16 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/kubernetes/addons/calico/calico-rbac.yml]
Oct 26 14:00:16 ctl01 salt-minion[4526]: [INFO    ] File /etc/kubernetes/addons/calico/calico-rbac.yml is in the correct state
Oct 26 14:00:16 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/addons/calico/calico-rbac.yml] at time 14:00:16.121779 duration_in_ms=28.756
Oct 26 14:00:16 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/addons/netchecker/netchecker-svc.yml] at time 14:00:16.122198
Oct 26 14:00:16 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/kubernetes/addons/netchecker/netchecker-svc.yml]
Oct 26 14:00:16 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:16 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:16 ctl01 salt-minion[4526]: [INFO    ] File /etc/kubernetes/addons/netchecker/netchecker-svc.yml is in the correct state
Oct 26 14:00:16 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/addons/netchecker/netchecker-svc.yml] at time 14:00:16.470222 duration_in_ms=348.023
Oct 26 14:00:16 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/addons/netchecker/netchecker-server.yml] at time 14:00:16.470844
Oct 26 14:00:16 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/kubernetes/addons/netchecker/netchecker-server.yml]
Oct 26 14:00:16 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:16 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:16 ctl01 kube-proxy[8230]: I1026 14:00:16.821011    8230 proxier.go:645] Not syncing iptables until Services and Endpoints have been received from master
Oct 26 14:00:16 ctl01 kube-proxy[8230]: E1026 14:00:16.850114    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: Get https://172.16.10.36:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:16 ctl01 kube-proxy[8230]: E1026 14:00:16.886509    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:16 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:17 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:17 ctl01 salt-minion[4526]: [INFO    ] File /etc/kubernetes/addons/netchecker/netchecker-server.yml is in the correct state
Oct 26 14:00:17 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/addons/netchecker/netchecker-server.yml] at time 14:00:17.207478 duration_in_ms=736.633
Oct 26 14:00:17 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/addons/netchecker/netchecker-agent.yml] at time 14:00:17.207951
Oct 26 14:00:17 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/kubernetes/addons/netchecker/netchecker-agent.yml]
Oct 26 14:00:17 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:17 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:17 ctl01 salt-minion[4526]: [INFO    ] File /etc/kubernetes/addons/netchecker/netchecker-agent.yml is in the correct state
Oct 26 14:00:17 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/addons/netchecker/netchecker-agent.yml] at time 14:00:17.611536 duration_in_ms=403.583
Oct 26 14:00:17 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/addons/netchecker/netchecker-serviceaccount.yml] at time 14:00:17.612444
Oct 26 14:00:17 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/kubernetes/addons/netchecker/netchecker-serviceaccount.yml]
Oct 26 14:00:17 ctl01 salt-minion[4526]: [INFO    ] File /etc/kubernetes/addons/netchecker/netchecker-serviceaccount.yml is in the correct state
Oct 26 14:00:17 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/addons/netchecker/netchecker-serviceaccount.yml] at time 14:00:17.638807 duration_in_ms=26.363
Oct 26 14:00:17 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/addons/netchecker/netchecker-roles.yml] at time 14:00:17.639115
Oct 26 14:00:17 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/kubernetes/addons/netchecker/netchecker-roles.yml]
Oct 26 14:00:17 ctl01 salt-minion[4526]: [INFO    ] File /etc/kubernetes/addons/netchecker/netchecker-roles.yml is in the correct state
Oct 26 14:00:17 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/addons/netchecker/netchecker-roles.yml] at time 14:00:17.661726 duration_in_ms=22.611
Oct 26 14:00:17 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/addons/prometheus/prometheus-roles.yml] at time 14:00:17.662026
Oct 26 14:00:17 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/kubernetes/addons/prometheus/prometheus-roles.yml]
Oct 26 14:00:17 ctl01 salt-minion[4526]: [INFO    ] File /etc/kubernetes/addons/prometheus/prometheus-roles.yml is in the correct state
Oct 26 14:00:17 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/addons/prometheus/prometheus-roles.yml] at time 14:00:17.683152 duration_in_ms=21.125
Oct 26 14:00:17 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/addons/dns] at time 14:00:17.683501
Oct 26 14:00:17 ctl01 salt-minion[4526]: [INFO    ] Executing state file.absent for [/etc/kubernetes/addons/dns]
Oct 26 14:00:17 ctl01 salt-minion[4526]: [INFO    ] {'removed': '/etc/kubernetes/addons/dns'}
Oct 26 14:00:17 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/addons/dns] at time 14:00:17.684899 duration_in_ms=1.398
Oct 26 14:00:17 ctl01 salt-minion[4526]: [INFO    ] Running state [kubectl -n kube-system delete svc kube-dns > /dev/null || echo "kube-dns is absent. OK" && true] at time 14:00:17.686493
Oct 26 14:00:17 ctl01 salt-minion[4526]: [INFO    ] Executing state cmd.run for [kubectl -n kube-system delete svc kube-dns > /dev/null || echo "kube-dns is absent. OK" && true]
Oct 26 14:00:17 ctl01 salt-minion[4526]: [INFO    ] Executing command 'kubectl -n kube-system delete svc kube-dns > /dev/null || echo "kube-dns is absent. OK" && true' in directory '/root'
Oct 26 14:00:17 ctl01 salt-minion[4526]: [INFO    ] {'pid': 10225, 'retcode': 0, 'stderr': 'The connection to the server localhost:8080 was refused - did you specify the right host or port?', 'stdout': 'kube-dns is absent. OK'}
Oct 26 14:00:17 ctl01 salt-minion[4526]: [INFO    ] Completed state [kubectl -n kube-system delete svc kube-dns > /dev/null || echo "kube-dns is absent. OK" && true] at time 14:00:17.815448 duration_in_ms=128.954
Oct 26 14:00:17 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/addons/coredns/coredns-cm.yml] at time 14:00:17.815922
Oct 26 14:00:17 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/kubernetes/addons/coredns/coredns-cm.yml]
Oct 26 14:00:17 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/kube-addons/coredns/coredns-cm.yml'
Oct 26 14:00:17 ctl01 kube-proxy[8230]: E1026 14:00:17.850721    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: Get https://172.16.10.36:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:17 ctl01 kube-proxy[8230]: E1026 14:00:17.887061    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:17 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:18 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:18 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:18 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:18 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 14:00:18 ctl01 salt-minion[4526]: New file
Oct 26 14:00:18 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/addons/coredns/coredns-cm.yml] at time 14:00:18.492902 duration_in_ms=676.979
Oct 26 14:00:18 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/addons/coredns/coredns-deploy.yml] at time 14:00:18.493332
Oct 26 14:00:18 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/kubernetes/addons/coredns/coredns-deploy.yml]
Oct 26 14:00:18 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/kube-addons/coredns/coredns-deploy.yml'
Oct 26 14:00:18 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:18 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:18 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 14:00:18 ctl01 salt-minion[4526]: New file
Oct 26 14:00:18 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/addons/coredns/coredns-deploy.yml] at time 14:00:18.832445 duration_in_ms=339.111
Oct 26 14:00:18 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/addons/coredns/coredns-svc.yml] at time 14:00:18.832888
Oct 26 14:00:18 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/kubernetes/addons/coredns/coredns-svc.yml]
Oct 26 14:00:18 ctl01 kube-proxy[8230]: E1026 14:00:18.851447    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: Get https://172.16.10.36:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:18 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/kube-addons/coredns/coredns-svc.yml'
Oct 26 14:00:18 ctl01 kube-proxy[8230]: E1026 14:00:18.887558    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:18 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:19 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:19 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 14:00:19 ctl01 salt-minion[4526]: New file
Oct 26 14:00:19 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/addons/coredns/coredns-svc.yml] at time 14:00:19.232653 duration_in_ms=399.764
Oct 26 14:00:19 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/addons/coredns/coredns-rbac.yml] at time 14:00:19.233037
Oct 26 14:00:19 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/kubernetes/addons/coredns/coredns-rbac.yml]
Oct 26 14:00:19 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/kube-addons/coredns/coredns-rbac.yml'
Oct 26 14:00:19 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:19 ctl01 systemd[1]: kubelet.service: Service hold-off time over, scheduling restart.
Oct 26 14:00:19 ctl01 systemd[1]: Stopped Kubernetes Kubelet Server.
Oct 26 14:00:19 ctl01 systemd[1]: Started Kubernetes Kubelet Server.
Oct 26 14:00:19 ctl01 kubelet[10393]: Flag --pod-manifest-path has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 26 14:00:19 ctl01 kubelet[10393]: Flag --address has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 26 14:00:19 ctl01 kubelet[10393]: Flag --allow-privileged has been deprecated, will be removed in a future version
Oct 26 14:00:19 ctl01 kubelet[10393]: Flag --cluster-dns has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 26 14:00:19 ctl01 kubelet[10393]: Flag --cluster-domain has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 26 14:00:19 ctl01 kubelet[10393]: Flag --fail-swap-on has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.440330   10393 flags.go:33] FLAG: --address="172.16.10.36"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.440400   10393 flags.go:33] FLAG: --allow-privileged="true"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.440410   10393 flags.go:33] FLAG: --allowed-unsafe-sysctls="[]"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.440421   10393 flags.go:33] FLAG: --alsologtostderr="false"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.440429   10393 flags.go:33] FLAG: --anonymous-auth="true"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.440435   10393 flags.go:33] FLAG: --application-metrics-count-limit="100"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.440442   10393 flags.go:33] FLAG: --authentication-token-webhook="false"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.440448   10393 flags.go:33] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.440456   10393 flags.go:33] FLAG: --authorization-mode="AlwaysAllow"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441067   10393 flags.go:33] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441097   10393 flags.go:33] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441104   10393 flags.go:33] FLAG: --azure-container-registry-config=""
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441110   10393 flags.go:33] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441117   10393 flags.go:33] FLAG: --bootstrap-checkpoint-path=""
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441123   10393 flags.go:33] FLAG: --bootstrap-kubeconfig=""
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441128   10393 flags.go:33] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441134   10393 flags.go:33] FLAG: --cgroup-driver="cgroupfs"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441140   10393 flags.go:33] FLAG: --cgroup-root=""
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441145   10393 flags.go:33] FLAG: --cgroups-per-qos="true"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441151   10393 flags.go:33] FLAG: --chaos-chance="0"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441161   10393 flags.go:33] FLAG: --client-ca-file=""
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441167   10393 flags.go:33] FLAG: --cloud-config=""
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441172   10393 flags.go:33] FLAG: --cloud-provider=""
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441177   10393 flags.go:33] FLAG: --cluster-dns="[10.254.0.10]"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441194   10393 flags.go:33] FLAG: --cluster-domain="cluster.local"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441200   10393 flags.go:33] FLAG: --cni-bin-dir="/opt/cni/bin"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441206   10393 flags.go:33] FLAG: --cni-conf-dir="/etc/cni/net.d"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441212   10393 flags.go:33] FLAG: --config=""
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441217   10393 flags.go:33] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441224   10393 flags.go:33] FLAG: --container-log-max-files="5"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441231   10393 flags.go:33] FLAG: --container-log-max-size="10Mi"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441237   10393 flags.go:33] FLAG: --container-runtime="remote"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441243   10393 flags.go:33] FLAG: --container-runtime-endpoint="unix:///run/containerd/containerd.sock"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441249   10393 flags.go:33] FLAG: --containerd="unix:///var/run/containerd.sock"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441256   10393 flags.go:33] FLAG: --containerized="false"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441261   10393 flags.go:33] FLAG: --contention-profiling="false"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441267   10393 flags.go:33] FLAG: --cpu-cfs-quota="true"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441272   10393 flags.go:33] FLAG: --cpu-cfs-quota-period="100ms"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441279   10393 flags.go:33] FLAG: --cpu-manager-policy="none"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441284   10393 flags.go:33] FLAG: --cpu-manager-reconcile-period="10s"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441290   10393 flags.go:33] FLAG: --docker="unix:///var/run/docker.sock"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441296   10393 flags.go:33] FLAG: --docker-endpoint="unix:///var/run/docker.sock"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441303   10393 flags.go:33] FLAG: --docker-env-metadata-whitelist=""
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441308   10393 flags.go:33] FLAG: --docker-only="false"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441314   10393 flags.go:33] FLAG: --docker-root="/var/lib/docker"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441320   10393 flags.go:33] FLAG: --docker-tls="false"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441326   10393 flags.go:33] FLAG: --docker-tls-ca="ca.pem"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441331   10393 flags.go:33] FLAG: --docker-tls-cert="cert.pem"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441375   10393 flags.go:33] FLAG: --docker-tls-key="key.pem"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441383   10393 flags.go:33] FLAG: --dynamic-config-dir=""
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441391   10393 flags.go:33] FLAG: --enable-controller-attach-detach="true"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441397   10393 flags.go:33] FLAG: --enable-debugging-handlers="true"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441403   10393 flags.go:33] FLAG: --enable-load-reader="false"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441409   10393 flags.go:33] FLAG: --enable-server="true"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441415   10393 flags.go:33] FLAG: --enforce-node-allocatable="[pods]"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441430   10393 flags.go:33] FLAG: --event-burst="10"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441437   10393 flags.go:33] FLAG: --event-qps="5"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441442   10393 flags.go:33] FLAG: --event-storage-age-limit="default=0"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441448   10393 flags.go:33] FLAG: --event-storage-event-limit="default=0"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441454   10393 flags.go:33] FLAG: --eviction-hard="imagefs.available<15%,memory.available<100Mi,nodefs.available<10%,nodefs.inodesFree<5%"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441472   10393 flags.go:33] FLAG: --eviction-max-pod-grace-period="0"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441479   10393 flags.go:33] FLAG: --eviction-minimum-reclaim=""
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441487   10393 flags.go:33] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441493   10393 flags.go:33] FLAG: --eviction-soft=""
Oct 26 14:00:19 ctl01 systemd[1]: Started Kubernetes systemd probe.
Oct 26 14:00:19 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441500   10393 flags.go:33] FLAG: --eviction-soft-grace-period=""
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441506   10393 flags.go:33] FLAG: --exit-on-lock-contention="false"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441512   10393 flags.go:33] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441518   10393 flags.go:33] FLAG: --experimental-bootstrap-kubeconfig=""
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441523   10393 flags.go:33] FLAG: --experimental-check-node-capabilities-before-mount="false"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441529   10393 flags.go:33] FLAG: --experimental-dockershim="false"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441535   10393 flags.go:33] FLAG: --experimental-dockershim-root-directory="/var/lib/dockershim"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441541   10393 flags.go:33] FLAG: --experimental-fail-swap-on="true"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441546   10393 flags.go:33] FLAG: --experimental-kernel-memcg-notification="false"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441552   10393 flags.go:33] FLAG: --experimental-mounter-path=""
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441557   10393 flags.go:33] FLAG: --fail-swap-on="true"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441563   10393 flags.go:33] FLAG: --feature-gates=""
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441571   10393 flags.go:33] FLAG: --file-check-frequency="20s"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441577   10393 flags.go:33] FLAG: --global-housekeeping-interval="1m0s"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441583   10393 flags.go:33] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441589   10393 flags.go:33] FLAG: --healthz-bind-address="127.0.0.1"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441606   10393 flags.go:33] FLAG: --healthz-port="10248"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441613   10393 flags.go:33] FLAG: --help="false"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441618   10393 flags.go:33] FLAG: --host-ipc-sources="[*]"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441626   10393 flags.go:33] FLAG: --host-network-sources="[*]"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441645   10393 flags.go:33] FLAG: --host-pid-sources="[*]"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441652   10393 flags.go:33] FLAG: --hostname-override="ctl01"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441658   10393 flags.go:33] FLAG: --housekeeping-interval="10s"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441664   10393 flags.go:33] FLAG: --http-check-frequency="20s"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441670   10393 flags.go:33] FLAG: --image-gc-high-threshold="85"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441676   10393 flags.go:33] FLAG: --image-gc-low-threshold="80"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441681   10393 flags.go:33] FLAG: --image-pull-progress-deadline="1m0s"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441687   10393 flags.go:33] FLAG: --image-service-endpoint=""
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441692   10393 flags.go:33] FLAG: --iptables-drop-bit="15"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441698   10393 flags.go:33] FLAG: --iptables-masquerade-bit="14"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441704   10393 flags.go:33] FLAG: --keep-terminated-pod-volumes="false"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441709   10393 flags.go:33] FLAG: --kube-api-burst="10"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441715   10393 flags.go:33] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441721   10393 flags.go:33] FLAG: --kube-api-qps="5"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441727   10393 flags.go:33] FLAG: --kube-reserved=""
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441741   10393 flags.go:33] FLAG: --kube-reserved-cgroup=""
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441746   10393 flags.go:33] FLAG: --kubeconfig="/etc/kubernetes/kubelet.kubeconfig"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441752   10393 flags.go:33] FLAG: --kubelet-cgroups=""
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441758   10393 flags.go:33] FLAG: --lock-file=""
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441763   10393 flags.go:33] FLAG: --log-backtrace-at=":0"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441770   10393 flags.go:33] FLAG: --log-cadvisor-usage="false"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441776   10393 flags.go:33] FLAG: --log-dir=""
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441781   10393 flags.go:33] FLAG: --log-file=""
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441786   10393 flags.go:33] FLAG: --log-flush-frequency="5s"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441792   10393 flags.go:33] FLAG: --logtostderr="true"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441798   10393 flags.go:33] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441804   10393 flags.go:33] FLAG: --make-iptables-util-chains="true"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441810   10393 flags.go:33] FLAG: --manifest-url=""
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441828   10393 flags.go:33] FLAG: --manifest-url-header=""
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441837   10393 flags.go:33] FLAG: --master-service-namespace="default"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441844   10393 flags.go:33] FLAG: --max-open-files="1000000"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441852   10393 flags.go:33] FLAG: --max-pods="110"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441858   10393 flags.go:33] FLAG: --maximum-dead-containers="-1"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441863   10393 flags.go:33] FLAG: --maximum-dead-containers-per-container="1"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441869   10393 flags.go:33] FLAG: --minimum-container-ttl-duration="0s"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441875   10393 flags.go:33] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441881   10393 flags.go:33] FLAG: --network-plugin="cni"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441886   10393 flags.go:33] FLAG: --network-plugin-mtu="0"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441892   10393 flags.go:33] FLAG: --node-ip="172.16.10.36"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441898   10393 flags.go:33] FLAG: --node-labels="node-role.kubernetes.io/master=true"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441909   10393 flags.go:33] FLAG: --node-status-max-images="50"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441915   10393 flags.go:33] FLAG: --node-status-update-frequency="10s"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441921   10393 flags.go:33] FLAG: --non-masquerade-cidr="10.0.0.0/8"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441926   10393 flags.go:33] FLAG: --oom-score-adj="-999"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441932   10393 flags.go:33] FLAG: --pod-cidr=""
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441938   10393 flags.go:33] FLAG: --pod-infra-container-image="docker-prod-local.artifactory.mirantis.com/mirantis/kubernetes/pause-amd64:v1.13.5-3"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441947   10393 flags.go:33] FLAG: --pod-manifest-path="/etc/kubernetes/manifests"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441953   10393 flags.go:33] FLAG: --pod-max-pids="-1"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441959   10393 flags.go:33] FLAG: --pods-per-core="0"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441965   10393 flags.go:33] FLAG: --port="10250"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441971   10393 flags.go:33] FLAG: --protect-kernel-defaults="false"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441976   10393 flags.go:33] FLAG: --provider-id=""
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441982   10393 flags.go:33] FLAG: --qos-reserved=""
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441988   10393 flags.go:33] FLAG: --read-only-port="10255"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441993   10393 flags.go:33] FLAG: --really-crash-for-testing="false"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.441999   10393 flags.go:33] FLAG: --redirect-container-streaming="false"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.442005   10393 flags.go:33] FLAG: --register-node="true"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.442010   10393 flags.go:33] FLAG: --register-schedulable="true"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.442016   10393 flags.go:33] FLAG: --register-with-taints=""
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.442023   10393 flags.go:33] FLAG: --registry-burst="10"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.442038   10393 flags.go:33] FLAG: --registry-qps="5"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.442044   10393 flags.go:33] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.442059   10393 flags.go:33] FLAG: --root-dir="/var/lib/kubelet"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.442065   10393 flags.go:33] FLAG: --rotate-certificates="false"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.442070   10393 flags.go:33] FLAG: --rotate-server-certificates="false"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.442076   10393 flags.go:33] FLAG: --runonce="false"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.442081   10393 flags.go:33] FLAG: --runtime-cgroups=""
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.442087   10393 flags.go:33] FLAG: --runtime-request-timeout="2m0s"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.442092   10393 flags.go:33] FLAG: --seccomp-profile-root="/var/lib/kubelet/seccomp"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.442098   10393 flags.go:33] FLAG: --serialize-image-pulls="true"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.442104   10393 flags.go:33] FLAG: --stderrthreshold="2"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.442109   10393 flags.go:33] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.442115   10393 flags.go:33] FLAG: --storage-driver-db="cadvisor"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.442121   10393 flags.go:33] FLAG: --storage-driver-host="localhost:8086"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.442127   10393 flags.go:33] FLAG: --storage-driver-password="root"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.442133   10393 flags.go:33] FLAG: --storage-driver-secure="false"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.442138   10393 flags.go:33] FLAG: --storage-driver-table="stats"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.442144   10393 flags.go:33] FLAG: --storage-driver-user="root"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.442149   10393 flags.go:33] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.442155   10393 flags.go:33] FLAG: --sync-frequency="1m0s"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.442161   10393 flags.go:33] FLAG: --system-cgroups=""
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.442166   10393 flags.go:33] FLAG: --system-reserved=""
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.442172   10393 flags.go:33] FLAG: --system-reserved-cgroup=""
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.442177   10393 flags.go:33] FLAG: --tls-cert-file=""
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.442183   10393 flags.go:33] FLAG: --tls-cipher-suites="[]"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.442192   10393 flags.go:33] FLAG: --tls-min-version=""
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.442198   10393 flags.go:33] FLAG: --tls-private-key-file=""
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.442203   10393 flags.go:33] FLAG: --v="2"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.442214   10393 flags.go:33] FLAG: --version="false"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.442222   10393 flags.go:33] FLAG: --vmodule=""
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.442229   10393 flags.go:33] FLAG: --volume-plugin-dir="/usr/libexec/kubernetes/kubelet-plugins/volume/exec/"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.442235   10393 flags.go:33] FLAG: --volume-stats-agg-period="1m0s"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.442285   10393 feature_gate.go:206] feature gates: &{map[]}
Oct 26 14:00:19 ctl01 kubelet[10393]: W1026 14:00:19.442307   10393 options.go:265] unknown 'kubernetes.io' or 'k8s.io' labels specified with --node-labels: [node-role.kubernetes.io/master]
Oct 26 14:00:19 ctl01 kubelet[10393]: W1026 14:00:19.442321   10393 options.go:266] in 1.15, --node-labels in the 'kubernetes.io' namespace must begin with an allowed prefix (kubelet.kubernetes.io, node.kubernetes.io) or be in the specifically allowed set (beta.kubernetes.io/arch, beta.kubernetes.io/instance-type, beta.kubernetes.io/os, failure-domain.beta.kubernetes.io/region, failure-domain.beta.kubernetes.io/zone, failure-domain.kubernetes.io/region, failure-domain.kubernetes.io/zone, kubernetes.io/arch, kubernetes.io/hostname, kubernetes.io/instance-type, kubernetes.io/os)
Oct 26 14:00:19 ctl01 kubelet[10393]: W1026 14:00:19.442340   10393 server.go:182] Warning: For remote container runtime, --pod-infra-container-image is ignored in kubelet, which should be set in that remote runtime instead
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.442394   10393 feature_gate.go:206] feature gates: &{map[]}
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.447505   10393 mount_linux.go:180] Detected OS with systemd
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.447546   10393 server.go:407] Version: v1.13.5-3+98374c02d2d8c1
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.447582   10393 feature_gate.go:206] feature gates: &{map[]}
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.447630   10393 feature_gate.go:206] feature gates: &{map[]}
Oct 26 14:00:19 ctl01 kubelet[10393]: W1026 14:00:19.447649   10393 options.go:265] unknown 'kubernetes.io' or 'k8s.io' labels specified with --node-labels: [node-role.kubernetes.io/master]
Oct 26 14:00:19 ctl01 kubelet[10393]: W1026 14:00:19.447662   10393 options.go:266] in 1.15, --node-labels in the 'kubernetes.io' namespace must begin with an allowed prefix (kubelet.kubernetes.io, node.kubernetes.io) or be in the specifically allowed set (beta.kubernetes.io/arch, beta.kubernetes.io/instance-type, beta.kubernetes.io/os, failure-domain.beta.kubernetes.io/region, failure-domain.beta.kubernetes.io/zone, failure-domain.kubernetes.io/region, failure-domain.kubernetes.io/zone, kubernetes.io/arch, kubernetes.io/hostname, kubernetes.io/instance-type, kubernetes.io/os)
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.447738   10393 plugins.go:103] No cloud provider specified.
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.447754   10393 server.go:523] No cloud provider specified: "" from the config file: ""
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.457134   10393 manager.go:155] cAdvisor running in container: "/sys/fs/cgroup/cpu,cpuacct/system.slice/kubelet.service"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.458125   10393 fs.go:142] Filesystem UUIDs: map[2019-10-26-15-49-44-00:/dev/sr0 3dc7f99a-1e06-428f-86bf-f1f8932b4515:/dev/vda1]
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.458142   10393 fs.go:143] Filesystem partitions: map[tmpfs:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /dev/vda1:{mountpoint:/ major:252 minor:1 fsType:ext4 blockSize:0}]
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.467686   10393 manager.go:229] Machine: {NumCores:8 CpuFrequency:2799994 MemoryCapacity:14705016832 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:70a4fc47e8254bb0a9da2f2beeb1a9ad SystemUUID:B60BACF9-7C5C-402C-99A3-F64133E62AD6 BootID:f6b0e736-be69-43fe-8cfa-5f08c44a9aa3 Filesystems:[{Device:tmpfs DeviceMajor:0 DeviceMinor:24 Capacity:1470504960 Type:vfs Inodes:1795046 HasInodes:true} {Device:/dev/vda1 DeviceMajor:252 DeviceMinor:1 Capacity:104022159360 Type:vfs Inodes:12800000 HasInodes:true}] DiskMap:map[252:0:{Name:vda Major:252 Minor:0 Size:107374182400 Scheduler:none}] NetworkDevices:[{Name:br-mgmt MacAddress:52:54:00:cc:bf:d7 Speed:0 Mtu:9000} {Name:ens3 MacAddress:52:54:00:d1:44:f2 Speed:-1 Mtu:9000} {Name:ens4 MacAddress:52:54:00:cc:bf:d7 Speed:-1 Mtu:9000} {Name:ens5 MacAddress:52:54:00:dd:4a:72 Speed:-1 Mtu:9000} {Name:ens5.1000 MacAddress:52:54:00:dd:4a:72 Speed:-1 Mtu:9000} {Name:ens6 MacAddress:52:54:00:23:fb:5b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:14705016832 Cores:[{Id:0 Threads:[0] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]} {Id:1 Memory:0 Cores:[{Id:0 Threads:[1] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]} {Id:2 Memory:0 Cores:[{Id:0 Threads:[2] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]} {Id:3 Memory:0 Cores:[{Id:0 Threads:[3] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]} {Id:4 Memory:0 Cores:[{Id:0 Threads:[4] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]} {Id:5 Memory:0 Cores:[{Id:0 Threads:[5] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]} {Id:6 Memory:0 Cores:
Oct 26 14:00:19 ctl01 kubelet[10393]: [{Id:0 Threads:[6] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]} {Id:7 Memory:0 Cores:[{Id:0 Threads:[7] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.468485   10393 manager.go:235] Version: {KernelVersion:4.15.0-66-generic ContainerOsVersion:Ubuntu 16.04.6 LTS DockerVersion:Unknown DockerAPIVersion:Unknown CadvisorVersion: CadvisorRevision:}
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.468703   10393 server.go:666] --cgroups-per-qos enabled, but --cgroup-root was not specified.  defaulting to /
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.469322   10393 container_manager_linux.go:248] container manager verified user specified cgroup-root exists: []
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.469391   10393 container_manager_linux.go:253] Creating Container Manager object based on Node Config: {RuntimeCgroupsName: SystemCgroupsName: KubeletCgroupsName: ContainerRuntime:remote CgroupsPerQOS:true CgroupRoot:/ CgroupDriver:cgroupfs KubeletRootDir:/var/lib/kubelet ProtectKernelDefaults:false NodeAllocatableConfig:{KubeReservedCgroupName: SystemReservedCgroupName: EnforceNodeAllocatable:map[pods:{}] KubeReserved:map[] SystemReserved:map[] HardEvictionThresholds:[{Signal:memory.available Operator:LessThan Value:{Quantity:100Mi Percentage:0} GracePeriod:0s MinReclaim:<nil>} {Signal:nodefs.available Operator:LessThan Value:{Quantity:<nil> Percentage:0.1} GracePeriod:0s MinReclaim:<nil>} {Signal:nodefs.inodesFree Operator:LessThan Value:{Quantity:<nil> Percentage:0.05} GracePeriod:0s MinReclaim:<nil>} {Signal:imagefs.available Operator:LessThan Value:{Quantity:<nil> Percentage:0.15} GracePeriod:0s MinReclaim:<nil>}]} QOSReserved:map[] ExperimentalCPUManagerPolicy:none ExperimentalCPUManagerReconcilePeriod:10s ExperimentalPodPidsLimit:-1 EnforceCPULimits:true CPUCFSQuotaPeriod:100ms}
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.469707   10393 container_manager_linux.go:272] Creating device plugin manager: true
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.469734   10393 manager.go:109] Creating Device Plugin manager at /var/lib/kubelet/device-plugins/kubelet.sock
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.469819   10393 state_mem.go:36] [cpumanager] initializing new in-memory state store
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.470036   10393 state_mem.go:84] [cpumanager] updated default cpuset: ""
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.470066   10393 state_mem.go:92] [cpumanager] updated cpuset assignments: "map[]"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.470088   10393 state_checkpoint.go:100] [cpumanager] state checkpoint: restored state from checkpoint
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.470102   10393 state_checkpoint.go:101] [cpumanager] state checkpoint: defaultCPUSet:
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.470245   10393 server.go:941] Using root directory: /var/lib/kubelet
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.470292   10393 kubelet.go:281] Adding pod path: /etc/kubernetes/manifests
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.470378   10393 file.go:68] Watching path "/etc/kubernetes/manifests"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.470458   10393 kubelet.go:306] Watching apiserver
Oct 26 14:00:19 ctl01 kubelet[10393]: E1026 14:00:19.471729   10393 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://172.16.10.36:443/api/v1/pods?fieldSelector=spec.nodeName%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:19 ctl01 kubelet[10393]: E1026 14:00:19.471748   10393 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://172.16.10.36:443/api/v1/nodes?fieldSelector=metadata.name%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:19 ctl01 kubelet[10393]: E1026 14:00:19.473668   10393 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.475492   10393 kuberuntime_manager.go:198] Container runtime containerd initialized, version: 1.2.5-3, apiVersion: v1alpha2
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.475970   10393 plugins.go:547] Loaded volume plugin "kubernetes.io/aws-ebs"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.475989   10393 plugins.go:547] Loaded volume plugin "kubernetes.io/empty-dir"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.476000   10393 plugins.go:547] Loaded volume plugin "kubernetes.io/gce-pd"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.476010   10393 plugins.go:547] Loaded volume plugin "kubernetes.io/git-repo"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.476020   10393 plugins.go:547] Loaded volume plugin "kubernetes.io/host-path"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.476030   10393 plugins.go:547] Loaded volume plugin "kubernetes.io/nfs"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.476040   10393 plugins.go:547] Loaded volume plugin "kubernetes.io/secret"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.476050   10393 plugins.go:547] Loaded volume plugin "kubernetes.io/iscsi"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.476063   10393 plugins.go:547] Loaded volume plugin "kubernetes.io/glusterfs"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.476079   10393 plugins.go:547] Loaded volume plugin "kubernetes.io/rbd"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.476090   10393 plugins.go:547] Loaded volume plugin "kubernetes.io/cinder"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.476100   10393 plugins.go:547] Loaded volume plugin "kubernetes.io/quobyte"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.476117   10393 plugins.go:547] Loaded volume plugin "kubernetes.io/cephfs"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.476130   10393 plugins.go:547] Loaded volume plugin "kubernetes.io/downward-api"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.476140   10393 plugins.go:547] Loaded volume plugin "kubernetes.io/fc"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.476149   10393 plugins.go:547] Loaded volume plugin "kubernetes.io/flocker"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.476159   10393 plugins.go:547] Loaded volume plugin "kubernetes.io/azure-file"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.476175   10393 plugins.go:547] Loaded volume plugin "kubernetes.io/configmap"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.476186   10393 plugins.go:547] Loaded volume plugin "kubernetes.io/vsphere-volume"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.476196   10393 plugins.go:547] Loaded volume plugin "kubernetes.io/azure-disk"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.476205   10393 plugins.go:547] Loaded volume plugin "kubernetes.io/photon-pd"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.476216   10393 plugins.go:547] Loaded volume plugin "kubernetes.io/projected"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.476235   10393 plugins.go:547] Loaded volume plugin "kubernetes.io/portworx-volume"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.476246   10393 plugins.go:547] Loaded volume plugin "kubernetes.io/scaleio"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.476256   10393 plugins.go:547] Loaded volume plugin "kubernetes.io/local-volume"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.476265   10393 plugins.go:547] Loaded volume plugin "kubernetes.io/storageos"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.476279   10393 plugins.go:547] Loaded volume plugin "kubernetes.io/csi"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.476535   10393 server.go:999] Started kubelet
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.476886   10393 server.go:157] Starting to listen read-only on 172.16.10.36:10255
Oct 26 14:00:19 ctl01 kubelet[10393]: E1026 14:00:19.477007   10393 event.go:212] Unable to write event: 'Post https://172.16.10.36:443/api/v1/namespaces/default/events: dial tcp 172.16.10.36:443: connect: connection refused' (may retry after sleeping)
Oct 26 14:00:19 ctl01 kubelet[10393]: E1026 14:00:19.477548   10393 cri_stats_provider.go:320] Failed to get the info of the filesystem with mountpoint "/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs": unable to find data in memory cache.
Oct 26 14:00:19 ctl01 kubelet[10393]: E1026 14:00:19.477572   10393 kubelet.go:1308] Image garbage collection failed once. Stats initialization may not have completed yet: invalid capacity 0 on image filesystem
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.477577   10393 fs_resource_analyzer.go:66] Starting FS ResourceAnalyzer
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.477616   10393 status_manager.go:152] Starting to sync pod status with apiserver
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.477629   10393 kubelet.go:1829] Starting kubelet main sync loop.
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.477642   10393 kubelet.go:1846] skipping pod synchronization - [container runtime status check may not have completed yet PLEG is not healthy: pleg has yet to be successful]
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.477818   10393 volume_manager.go:246] The desired_state_of_world populator starts
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.477828   10393 volume_manager.go:248] Starting Kubelet Volume Manager
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.478310   10393 desired_state_of_world_populator.go:130] Desired state populator starts to run
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.476618   10393 server.go:137] Starting to listen on 172.16.10.36:10250
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.494478   10393 factory.go:136] Registering containerd factory
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.495448   10393 server.go:333] Adding debug handlers to kubelet server.
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.495907   10393 factory.go:54] Registering systemd factory
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.496254   10393 factory.go:97] Registering Raw factory
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.496497   10393 manager.go:1222] Started watching for new ooms in manager
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.500356   10393 manager.go:365] Starting recovery of all containers
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.562501   10393 manager.go:370] Recovery completed
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.577769   10393 kubelet.go:1846] skipping pod synchronization - [container runtime status check may not have completed yet]
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.578027   10393 kubelet_node_status.go:279] Setting node annotation to enable volume controller attach/detach
Oct 26 14:00:19 ctl01 kubelet[10393]: E1026 14:00:19.578034   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.578345   10393 setters.go:72] Using node IP: "172.16.10.36"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.579303   10393 kubelet_node_status.go:447] Recording NodeHasSufficientMemory event message for node ctl01
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.579338   10393 kubelet_node_status.go:447] Recording NodeHasNoDiskPressure event message for node ctl01
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.579361   10393 kubelet_node_status.go:447] Recording NodeHasSufficientPID event message for node ctl01
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.579387   10393 kubelet_node_status.go:72] Attempting to register node ctl01
Oct 26 14:00:19 ctl01 kubelet[10393]: E1026 14:00:19.579775   10393 kubelet_node_status.go:94] Unable to register node "ctl01" with API server: Post https://172.16.10.36:443/api/v1/nodes: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:19 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 14:00:19 ctl01 salt-minion[4526]: New file
Oct 26 14:00:19 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/addons/coredns/coredns-rbac.yml] at time 14:00:19.603860 duration_in_ms=370.821
Oct 26 14:00:19 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/addons/metrics-server] at time 14:00:19.604234
Oct 26 14:00:19 ctl01 salt-minion[4526]: [INFO    ] Executing state file.absent for [/etc/kubernetes/addons/metrics-server]
Oct 26 14:00:19 ctl01 salt-minion[4526]: [INFO    ] File /etc/kubernetes/addons/metrics-server is not present
Oct 26 14:00:19 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/addons/metrics-server] at time 14:00:19.605329 duration_in_ms=1.095
Oct 26 14:00:19 ctl01 salt-minion[4526]: [INFO    ] Running state [/srv/kubernetes/known_tokens.csv] at time 14:00:19.605609
Oct 26 14:00:19 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/srv/kubernetes/known_tokens.csv]
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.623941   10393 kubelet_node_status.go:279] Setting node annotation to enable volume controller attach/detach
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.624343   10393 setters.go:72] Using node IP: "172.16.10.36"
Oct 26 14:00:19 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/known_tokens.csv'
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.625840   10393 kubelet_node_status.go:447] Recording NodeHasSufficientMemory event message for node ctl01
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.626045   10393 kubelet_node_status.go:447] Recording NodeHasNoDiskPressure event message for node ctl01
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.626088   10393 kubelet_node_status.go:447] Recording NodeHasSufficientPID event message for node ctl01
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.626181   10393 cpu_manager.go:155] [cpumanager] starting with none policy
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.626199   10393 cpu_manager.go:156] [cpumanager] reconciling every 10s
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.626228   10393 policy_none.go:42] [cpumanager] none policy: Start
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.629596   10393 manager.go:196] Starting Device Plugin manager
Oct 26 14:00:19 ctl01 kubelet[10393]: W1026 14:00:19.629652   10393 manager.go:537] Failed to retrieve checkpoint for "kubelet_internal_checkpoint": checkpoint is not found
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.629905   10393 manager.go:231] Serving device plugin registration server on "/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.629994   10393 plugin_watcher.go:90] Plugin Watcher Start at /var/lib/kubelet/plugins_registry
Oct 26 14:00:19 ctl01 kubelet[10393]: E1026 14:00:19.630416   10393 eviction_manager.go:247] eviction manager: failed to get summary stats: failed to get node info: node "ctl01" not found
Oct 26 14:00:19 ctl01 kubelet[10393]: E1026 14:00:19.678312   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:19 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.777954   10393 kubelet.go:1908] SyncLoop (ADD, "file"): ""
Oct 26 14:00:19 ctl01 kubelet[10393]: E1026 14:00:19.778460   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.780008   10393 kubelet_node_status.go:279] Setting node annotation to enable volume controller attach/detach
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.780318   10393 setters.go:72] Using node IP: "172.16.10.36"
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.781313   10393 kubelet_node_status.go:447] Recording NodeHasSufficientMemory event message for node ctl01
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.781387   10393 kubelet_node_status.go:447] Recording NodeHasNoDiskPressure event message for node ctl01
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.781405   10393 kubelet_node_status.go:447] Recording NodeHasSufficientPID event message for node ctl01
Oct 26 14:00:19 ctl01 kubelet[10393]: I1026 14:00:19.781429   10393 kubelet_node_status.go:72] Attempting to register node ctl01
Oct 26 14:00:19 ctl01 kubelet[10393]: E1026 14:00:19.781835   10393 kubelet_node_status.go:94] Unable to register node "ctl01" with API server: Post https://172.16.10.36:443/api/v1/nodes: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:19 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:19 ctl01 kube-proxy[8230]: E1026 14:00:19.852385    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: Get https://172.16.10.36:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:19 ctl01 kubelet[10393]: E1026 14:00:19.878592   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:19 ctl01 kube-proxy[8230]: E1026 14:00:19.888031    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:19 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:19 ctl01 kubelet[10393]: E1026 14:00:19.978787   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:20 ctl01 kubelet[10393]: E1026 14:00:20.079010   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:20 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:20 ctl01 kubelet[10393]: E1026 14:00:20.179192   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:20 ctl01 kubelet[10393]: I1026 14:00:20.181989   10393 kubelet_node_status.go:279] Setting node annotation to enable volume controller attach/detach
Oct 26 14:00:20 ctl01 kubelet[10393]: I1026 14:00:20.182379   10393 setters.go:72] Using node IP: "172.16.10.36"
Oct 26 14:00:20 ctl01 kubelet[10393]: I1026 14:00:20.183541   10393 kubelet_node_status.go:447] Recording NodeHasSufficientMemory event message for node ctl01
Oct 26 14:00:20 ctl01 kubelet[10393]: I1026 14:00:20.183580   10393 kubelet_node_status.go:447] Recording NodeHasNoDiskPressure event message for node ctl01
Oct 26 14:00:20 ctl01 kubelet[10393]: I1026 14:00:20.183594   10393 kubelet_node_status.go:447] Recording NodeHasSufficientPID event message for node ctl01
Oct 26 14:00:20 ctl01 kubelet[10393]: I1026 14:00:20.183618   10393 kubelet_node_status.go:72] Attempting to register node ctl01
Oct 26 14:00:20 ctl01 kubelet[10393]: E1026 14:00:20.184009   10393 kubelet_node_status.go:94] Unable to register node "ctl01" with API server: Post https://172.16.10.36:443/api/v1/nodes: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:20 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 14:00:20 ctl01 salt-minion[4526]: New file
Oct 26 14:00:20 ctl01 salt-minion[4526]: [INFO    ] Completed state [/srv/kubernetes/known_tokens.csv] at time 14:00:20.263373 duration_in_ms=657.763
Oct 26 14:00:20 ctl01 salt-minion[4526]: [INFO    ] Running state [/srv/kubernetes/basic_auth.csv] at time 14:00:20.263732
Oct 26 14:00:20 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/srv/kubernetes/basic_auth.csv]
Oct 26 14:00:20 ctl01 kubelet[10393]: E1026 14:00:20.279525   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:20 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/basic_auth.csv'
Oct 26 14:00:20 ctl01 kubelet[10393]: E1026 14:00:20.379799   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:20 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:20 ctl01 kubelet[10393]: E1026 14:00:20.472545   10393 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://172.16.10.36:443/api/v1/pods?fieldSelector=spec.nodeName%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:20 ctl01 kubelet[10393]: E1026 14:00:20.473545   10393 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://172.16.10.36:443/api/v1/nodes?fieldSelector=metadata.name%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:20 ctl01 kubelet[10393]: E1026 14:00:20.474446   10393 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:20 ctl01 kubelet[10393]: E1026 14:00:20.479981   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:20 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:20 ctl01 kubelet[10393]: E1026 14:00:20.580242   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:20 ctl01 kubelet[10393]: E1026 14:00:20.639561   10393 event.go:212] Unable to write event: 'Post https://172.16.10.36:443/api/v1/namespaces/default/events: dial tcp 172.16.10.36:443: connect: connection refused' (may retry after sleeping)
Oct 26 14:00:20 ctl01 kubelet[10393]: E1026 14:00:20.680501   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:20 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 14:00:20 ctl01 salt-minion[4526]: New file
Oct 26 14:00:20 ctl01 salt-minion[4526]: [INFO    ] Completed state [/srv/kubernetes/basic_auth.csv] at time 14:00:20.710001 duration_in_ms=446.267
Oct 26 14:00:20 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/default/kube-apiserver] at time 14:00:20.710336
Oct 26 14:00:20 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/default/kube-apiserver]
Oct 26 14:00:20 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 14:00:20 ctl01 salt-minion[4526]: New file
Oct 26 14:00:20 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/default/kube-apiserver] at time 14:00:20.712781 duration_in_ms=2.445
Oct 26 14:00:20 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/scheduler.kubeconfig] at time 14:00:20.713054
Oct 26 14:00:20 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/kubernetes/scheduler.kubeconfig]
Oct 26 14:00:20 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/kube-scheduler/scheduler.kubeconfig'
Oct 26 14:00:20 ctl01 kubelet[10393]: E1026 14:00:20.780702   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:20 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:20 ctl01 kube-proxy[8230]: E1026 14:00:20.853515    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: Get https://172.16.10.36:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:20 ctl01 kubelet[10393]: E1026 14:00:20.880966   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:20 ctl01 kube-proxy[8230]: E1026 14:00:20.888716    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:20 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:20 ctl01 kubelet[10393]: E1026 14:00:20.981195   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:20 ctl01 kubelet[10393]: I1026 14:00:20.984302   10393 kubelet_node_status.go:279] Setting node annotation to enable volume controller attach/detach
Oct 26 14:00:20 ctl01 kubelet[10393]: I1026 14:00:20.984761   10393 setters.go:72] Using node IP: "172.16.10.36"
Oct 26 14:00:20 ctl01 kubelet[10393]: I1026 14:00:20.986141   10393 kubelet_node_status.go:447] Recording NodeHasSufficientMemory event message for node ctl01
Oct 26 14:00:20 ctl01 kubelet[10393]: I1026 14:00:20.986208   10393 kubelet_node_status.go:447] Recording NodeHasNoDiskPressure event message for node ctl01
Oct 26 14:00:20 ctl01 kubelet[10393]: I1026 14:00:20.986234   10393 kubelet_node_status.go:447] Recording NodeHasSufficientPID event message for node ctl01
Oct 26 14:00:20 ctl01 kubelet[10393]: I1026 14:00:20.986274   10393 kubelet_node_status.go:72] Attempting to register node ctl01
Oct 26 14:00:20 ctl01 kubelet[10393]: E1026 14:00:20.986902   10393 kubelet_node_status.go:94] Unable to register node "ctl01" with API server: Post https://172.16.10.36:443/api/v1/nodes: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:21 ctl01 kubelet[10393]: E1026 14:00:21.081406   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:21 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:21 ctl01 kubelet[10393]: E1026 14:00:21.181594   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:21 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:21 ctl01 kubelet[10393]: E1026 14:00:21.281792   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:21 ctl01 kubelet[10393]: E1026 14:00:21.381970   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:21 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 14:00:21 ctl01 salt-minion[4526]: New file
Oct 26 14:00:21 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/scheduler.kubeconfig] at time 14:00:21.415263 duration_in_ms=702.206
Oct 26 14:00:21 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/controller-manager.kubeconfig] at time 14:00:21.415734
Oct 26 14:00:21 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/kubernetes/controller-manager.kubeconfig]
Oct 26 14:00:21 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/kube-controller-manager/controller-manager.kubeconfig'
Oct 26 14:00:21 ctl01 kubelet[10393]: E1026 14:00:21.473325   10393 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://172.16.10.36:443/api/v1/pods?fieldSelector=spec.nodeName%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:21 ctl01 kubelet[10393]: E1026 14:00:21.474411   10393 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://172.16.10.36:443/api/v1/nodes?fieldSelector=metadata.name%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:21 ctl01 kubelet[10393]: E1026 14:00:21.475492   10393 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:21 ctl01 kubelet[10393]: E1026 14:00:21.482153   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:21 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:21 ctl01 kubelet[10393]: E1026 14:00:21.582316   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:21 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:21 ctl01 kubelet[10393]: E1026 14:00:21.682503   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:21 ctl01 kubelet[10393]: E1026 14:00:21.782701   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:21 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:21 ctl01 kube-proxy[8230]: E1026 14:00:21.854520    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: Get https://172.16.10.36:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:21 ctl01 kubelet[10393]: E1026 14:00:21.882944   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:21 ctl01 kube-proxy[8230]: E1026 14:00:21.889404    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:21 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:21 ctl01 kubelet[10393]: E1026 14:00:21.983217   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:22 ctl01 kubelet[10393]: E1026 14:00:22.083396   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 14:00:22 ctl01 salt-minion[4526]: New file
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/controller-manager.kubeconfig] at time 14:00:22.107810 duration_in_ms=692.075
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/default/kube-controller-manager] at time 14:00:22.108152
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/default/kube-controller-manager]
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 14:00:22 ctl01 salt-minion[4526]: New file
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/default/kube-controller-manager] at time 14:00:22.110669 duration_in_ms=2.517
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/default/kube-scheduler] at time 14:00:22.110921
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/default/kube-scheduler]
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 14:00:22 ctl01 salt-minion[4526]: New file
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/default/kube-scheduler] at time 14:00:22.113169 duration_in_ms=2.248
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/systemd/system/kube-apiserver.service] at time 14:00:22.113423
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/systemd/system/kube-apiserver.service]
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/systemd/kube-apiserver.service'
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 14:00:22 ctl01 salt-minion[4526]: New file
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/systemd/system/kube-apiserver.service] at time 14:00:22.146050 duration_in_ms=32.626
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/systemd/system/kube-scheduler.service] at time 14:00:22.146365
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/systemd/system/kube-scheduler.service]
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/systemd/kube-scheduler.service'
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 14:00:22 ctl01 salt-minion[4526]: New file
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/systemd/system/kube-scheduler.service] at time 14:00:22.178757 duration_in_ms=32.391
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/systemd/system/kube-controller-manager.service] at time 14:00:22.178992
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/systemd/system/kube-controller-manager.service]
Oct 26 14:00:22 ctl01 kubelet[10393]: E1026 14:00:22.183657   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/systemd/kube-controller-manager.service'
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 14:00:22 ctl01 salt-minion[4526]: New file
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/systemd/system/kube-controller-manager.service] at time 14:00:22.212268 duration_in_ms=33.275
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/ssl/kubernetes-server.crt] at time 14:00:22.212707
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/kubernetes/ssl/kubernetes-server.crt]
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** '_certs/kubernetes/kubernetes-server.crt'
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 14:00:22 ctl01 salt-minion[4526]: New file
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/ssl/kubernetes-server.crt] at time 14:00:22.239584 duration_in_ms=26.877
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/ssl/kubernetes-server.key] at time 14:00:22.239968
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/kubernetes/ssl/kubernetes-server.key]
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** '_certs/kubernetes/kubernetes-server.key'
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 14:00:22 ctl01 salt-minion[4526]: New file
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/ssl/kubernetes-server.key] at time 14:00:22.263553 duration_in_ms=23.585
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/ssl/kubernetes-server.pem] at time 14:00:22.263865
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/kubernetes/ssl/kubernetes-server.pem]
Oct 26 14:00:22 ctl01 kubelet[10393]: E1026 14:00:22.283932   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** '_certs/kubernetes/kubernetes-server.pem'
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 14:00:22 ctl01 salt-minion[4526]: New file
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/ssl/kubernetes-server.pem] at time 14:00:22.288791 duration_in_ms=24.926
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] Running state [kube-apiserver] at time 14:00:22.296913
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] Executing state service.running for [kube-apiserver]
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'status', 'kube-apiserver.service', '-n', '0'] in directory '/root'
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-active', 'kube-apiserver.service'] in directory '/root'
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-enabled', 'kube-apiserver.service'] in directory '/root'
Oct 26 14:00:22 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemd-run', '--scope', 'systemctl', 'start', 'kube-apiserver.service'] in directory '/root'
Oct 26 14:00:22 ctl01 kubelet[10393]: E1026 14:00:22.384311   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:22 ctl01 systemd[1]: Started /bin/systemctl start kube-apiserver.service.
Oct 26 14:00:22 ctl01 systemd[1]: Starting Kubernetes API Server...
Oct 26 14:00:22 ctl01 kubelet[10393]: E1026 14:00:22.474215   10393 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://172.16.10.36:443/api/v1/pods?fieldSelector=spec.nodeName%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:22 ctl01 kubelet[10393]: E1026 14:00:22.475087   10393 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://172.16.10.36:443/api/v1/nodes?fieldSelector=metadata.name%3Dctl01&limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:22 ctl01 kubelet[10393]: E1026 14:00:22.476108   10393 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.16.10.36:443: connect: connection refused
Oct 26 14:00:22 ctl01 kubelet[10393]: E1026 14:00:22.484578   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: Flag --insecure-bind-address has been deprecated, This flag will be removed in a future version.
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: Flag --insecure-port has been deprecated, This flag will be removed in a future version.
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.545760   10762 flags.go:33] FLAG: --address="0.0.0.0"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.545829   10762 flags.go:33] FLAG: --admission-control="[]"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.545851   10762 flags.go:33] FLAG: --admission-control-config-file=""
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.545865   10762 flags.go:33] FLAG: --advertise-address="172.16.10.36"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.545879   10762 flags.go:33] FLAG: --allow-privileged="true"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.545893   10762 flags.go:33] FLAG: --alsologtostderr="false"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.545916   10762 flags.go:33] FLAG: --anonymous-auth="false"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.545926   10762 flags.go:33] FLAG: --api-audiences="[]"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.545939   10762 flags.go:33] FLAG: --apiserver-count="1"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.545953   10762 flags.go:33] FLAG: --application-metrics-count-limit="100"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.545963   10762 flags.go:33] FLAG: --audit-dynamic-configuration="false"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.545972   10762 flags.go:33] FLAG: --audit-log-batch-buffer-size="10000"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.545986   10762 flags.go:33] FLAG: --audit-log-batch-max-size="1"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546004   10762 flags.go:33] FLAG: --audit-log-batch-max-wait="0s"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546028   10762 flags.go:33] FLAG: --audit-log-batch-throttle-burst="0"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546048   10762 flags.go:33] FLAG: --audit-log-batch-throttle-enable="false"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546067   10762 flags.go:33] FLAG: --audit-log-batch-throttle-qps="0"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546094   10762 flags.go:33] FLAG: --audit-log-format="json"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546118   10762 flags.go:33] FLAG: --audit-log-maxage="0"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546137   10762 flags.go:33] FLAG: --audit-log-maxbackup="0"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546155   10762 flags.go:33] FLAG: --audit-log-maxsize="0"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546173   10762 flags.go:33] FLAG: --audit-log-mode="blocking"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546191   10762 flags.go:33] FLAG: --audit-log-path=""
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546209   10762 flags.go:33] FLAG: --audit-log-truncate-enabled="false"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546228   10762 flags.go:33] FLAG: --audit-log-truncate-max-batch-size="10485760"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546254   10762 flags.go:33] FLAG: --audit-log-truncate-max-event-size="102400"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546275   10762 flags.go:33] FLAG: --audit-log-version="audit.k8s.io/v1"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546304   10762 flags.go:33] FLAG: --audit-policy-file=""
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546314   10762 flags.go:33] FLAG: --audit-webhook-batch-buffer-size="10000"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546324   10762 flags.go:33] FLAG: --audit-webhook-batch-initial-backoff="10s"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546333   10762 flags.go:33] FLAG: --audit-webhook-batch-max-size="400"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546342   10762 flags.go:33] FLAG: --audit-webhook-batch-max-wait="30s"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546351   10762 flags.go:33] FLAG: --audit-webhook-batch-throttle-burst="15"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546371   10762 flags.go:33] FLAG: --audit-webhook-batch-throttle-enable="true"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546381   10762 flags.go:33] FLAG: --audit-webhook-batch-throttle-qps="10"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546391   10762 flags.go:33] FLAG: --audit-webhook-config-file=""
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546400   10762 flags.go:33] FLAG: --audit-webhook-initial-backoff="10s"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546409   10762 flags.go:33] FLAG: --audit-webhook-mode="batch"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546418   10762 flags.go:33] FLAG: --audit-webhook-truncate-enabled="false"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546427   10762 flags.go:33] FLAG: --audit-webhook-truncate-max-batch-size="10485760"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546436   10762 flags.go:33] FLAG: --audit-webhook-truncate-max-event-size="102400"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546446   10762 flags.go:33] FLAG: --audit-webhook-version="audit.k8s.io/v1"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546455   10762 flags.go:33] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546464   10762 flags.go:33] FLAG: --authentication-token-webhook-config-file=""
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546472   10762 flags.go:33] FLAG: --authorization-mode="[Node,RBAC]"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546489   10762 flags.go:33] FLAG: --authorization-policy-file=""
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546498   10762 flags.go:33] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546508   10762 flags.go:33] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546518   10762 flags.go:33] FLAG: --authorization-webhook-config-file=""
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546526   10762 flags.go:33] FLAG: --azure-container-registry-config=""
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546534   10762 flags.go:33] FLAG: --basic-auth-file="/srv/kubernetes/basic_auth.csv"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546545   10762 flags.go:33] FLAG: --bind-address="0.0.0.0"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546556   10762 flags.go:33] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546568   10762 flags.go:33] FLAG: --cert-dir="/var/run/kubernetes"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546580   10762 flags.go:33] FLAG: --client-ca-file="/etc/kubernetes/ssl/ca-kubernetes.crt"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546592   10762 flags.go:33] FLAG: --cloud-config=""
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546602   10762 flags.go:33] FLAG: --cloud-provider=""
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546611   10762 flags.go:33] FLAG: --cloud-provider-gce-lb-src-cidrs="130.211.0.0/22,209.85.152.0/22,209.85.204.0/22,35.191.0.0/16"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546629   10762 flags.go:33] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546643   10762 flags.go:33] FLAG: --containerd="unix:///var/run/containerd.sock"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546653   10762 flags.go:33] FLAG: --contention-profiling="false"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546662   10762 flags.go:33] FLAG: --cors-allowed-origins="[]"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546673   10762 flags.go:33] FLAG: --default-not-ready-toleration-seconds="300"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546683   10762 flags.go:33] FLAG: --default-unreachable-toleration-seconds="300"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546695   10762 flags.go:33] FLAG: --default-watch-cache-size="100"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546718   10762 flags.go:33] FLAG: --delete-collection-workers="1"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546730   10762 flags.go:33] FLAG: --deserialization-cache-size="0"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546742   10762 flags.go:33] FLAG: --disable-admission-plugins="[]"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546760   10762 flags.go:33] FLAG: --docker="unix:///var/run/docker.sock"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546772   10762 flags.go:33] FLAG: --docker-env-metadata-whitelist=""
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546782   10762 flags.go:33] FLAG: --docker-only="false"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546792   10762 flags.go:33] FLAG: --docker-root="/var/lib/docker"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546803   10762 flags.go:33] FLAG: --docker-tls="false"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546813   10762 flags.go:33] FLAG: --docker-tls-ca="ca.pem"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546823   10762 flags.go:33] FLAG: --docker-tls-cert="cert.pem"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546834   10762 flags.go:33] FLAG: --docker-tls-key="key.pem"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546844   10762 flags.go:33] FLAG: --enable-admission-plugins="[NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,DefaultStorageClass]"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546864   10762 flags.go:33] FLAG: --enable-aggregator-routing="false"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546874   10762 flags.go:33] FLAG: --enable-bootstrap-token-auth="false"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546883   10762 flags.go:33] FLAG: --enable-garbage-collector="true"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546893   10762 flags.go:33] FLAG: --enable-load-reader="false"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546902   10762 flags.go:33] FLAG: --enable-logs-handler="true"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546911   10762 flags.go:33] FLAG: --enable-swagger-ui="false"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546920   10762 flags.go:33] FLAG: --encryption-provider-config=""
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546929   10762 flags.go:33] FLAG: --endpoint-reconciler-type="lease"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546938   10762 flags.go:33] FLAG: --etcd-cafile="/var/lib/etcd/ca.pem"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546948   10762 flags.go:33] FLAG: --etcd-certfile="/var/lib/etcd/etcd-client.crt"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546958   10762 flags.go:33] FLAG: --etcd-compaction-interval="5m0s"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546968   10762 flags.go:33] FLAG: --etcd-count-metric-poll-period="1m0s"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546977   10762 flags.go:33] FLAG: --etcd-keyfile="/var/lib/etcd/etcd-client.key"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546988   10762 flags.go:33] FLAG: --etcd-prefix="/registry"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.546997   10762 flags.go:33] FLAG: --etcd-servers="[https://172.16.10.36:4001]"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547017   10762 flags.go:33] FLAG: --etcd-servers-overrides="[]"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547031   10762 flags.go:33] FLAG: --event-storage-age-limit="default=0"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547043   10762 flags.go:33] FLAG: --event-storage-event-limit="default=0"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547054   10762 flags.go:33] FLAG: --event-ttl="1h0m0s"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547066   10762 flags.go:33] FLAG: --experimental-encryption-provider-config=""
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547091   10762 flags.go:33] FLAG: --external-hostname=""
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547103   10762 flags.go:33] FLAG: --feature-gates=""
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547120   10762 flags.go:33] FLAG: --global-housekeeping-interval="1m0s"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547129   10762 flags.go:33] FLAG: --help="false"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547138   10762 flags.go:33] FLAG: --housekeeping-interval="10s"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547148   10762 flags.go:33] FLAG: --http2-max-streams-per-connection="0"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547160   10762 flags.go:33] FLAG: --insecure-bind-address="0.0.0.0"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547171   10762 flags.go:33] FLAG: --insecure-port="8080"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547182   10762 flags.go:33] FLAG: --kubelet-certificate-authority=""
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547192   10762 flags.go:33] FLAG: --kubelet-client-certificate=""
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547203   10762 flags.go:33] FLAG: --kubelet-client-key=""
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547214   10762 flags.go:33] FLAG: --kubelet-https="true"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547226   10762 flags.go:33] FLAG: --kubelet-port="10250"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547240   10762 flags.go:33] FLAG: --kubelet-preferred-address-types="[Hostname,InternalDNS,InternalIP,ExternalDNS,ExternalIP]"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547263   10762 flags.go:33] FLAG: --kubelet-read-only-port="10255"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547276   10762 flags.go:33] FLAG: --kubelet-timeout="5s"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547288   10762 flags.go:33] FLAG: --kubernetes-service-node-port="0"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547299   10762 flags.go:33] FLAG: --log-backtrace-at=":0"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547312   10762 flags.go:33] FLAG: --log-cadvisor-usage="false"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547323   10762 flags.go:33] FLAG: --log-dir=""
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547332   10762 flags.go:33] FLAG: --log-file=""
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547340   10762 flags.go:33] FLAG: --log-flush-frequency="5s"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547350   10762 flags.go:33] FLAG: --logtostderr="true"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547359   10762 flags.go:33] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547370   10762 flags.go:33] FLAG: --master-service-namespace="default"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547379   10762 flags.go:33] FLAG: --max-connection-bytes-per-sec="0"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547389   10762 flags.go:33] FLAG: --max-mutating-requests-inflight="200"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547399   10762 flags.go:33] FLAG: --max-requests-inflight="400"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547408   10762 flags.go:33] FLAG: --mesos-agent="127.0.0.1:5051"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547419   10762 flags.go:33] FLAG: --mesos-agent-timeout="10s"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547430   10762 flags.go:33] FLAG: --min-request-timeout="1800"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547441   10762 flags.go:33] FLAG: --oidc-ca-file=""
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547467   10762 flags.go:33] FLAG: --oidc-client-id=""
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547478   10762 flags.go:33] FLAG: --oidc-groups-claim=""
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547488   10762 flags.go:33] FLAG: --oidc-groups-prefix=""
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547499   10762 flags.go:33] FLAG: --oidc-issuer-url=""
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547509   10762 flags.go:33] FLAG: --oidc-required-claim=""
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547526   10762 flags.go:33] FLAG: --oidc-signing-algs="[RS256]"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547538   10762 flags.go:33] FLAG: --oidc-username-claim="sub"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547547   10762 flags.go:33] FLAG: --oidc-username-prefix=""
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547557   10762 flags.go:33] FLAG: --port="8080"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547568   10762 flags.go:33] FLAG: --profiling="true"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547578   10762 flags.go:33] FLAG: --proxy-client-cert-file="/etc/kubernetes/ssl/kube-aggregator-proxy-client.crt"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547591   10762 flags.go:33] FLAG: --proxy-client-key-file="/etc/kubernetes/ssl/kube-aggregator-proxy-client.key"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547604   10762 flags.go:33] FLAG: --repair-malformed-updates="false"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547615   10762 flags.go:33] FLAG: --request-timeout="1m0s"
Oct 26 14:00:22 ctl01 kubelet[10393]: E1026 14:00:22.584805   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:22 ctl01 kubelet[10393]: I1026 14:00:22.587120   10393 kubelet_node_status.go:279] Setting node annotation to enable volume controller attach/detach
Oct 26 14:00:22 ctl01 kubelet[10393]: I1026 14:00:22.587532   10393 setters.go:72] Using node IP: "172.16.10.36"
Oct 26 14:00:22 ctl01 kubelet[10393]: I1026 14:00:22.588699   10393 kubelet_node_status.go:447] Recording NodeHasSufficientMemory event message for node ctl01
Oct 26 14:00:22 ctl01 kubelet[10393]: I1026 14:00:22.588773   10393 kubelet_node_status.go:447] Recording NodeHasNoDiskPressure event message for node ctl01
Oct 26 14:00:22 ctl01 kubelet[10393]: I1026 14:00:22.588805   10393 kubelet_node_status.go:447] Recording NodeHasSufficientPID event message for node ctl01
Oct 26 14:00:22 ctl01 kubelet[10393]: I1026 14:00:22.588860   10393 kubelet_node_status.go:72] Attempting to register node ctl01
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547627   10762 flags.go:33] FLAG: --requestheader-allowed-names="[system:kube-controller-manager]"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547646   10762 flags.go:33] FLAG: --requestheader-client-ca-file="/etc/kubernetes/ssl/ca-kubernetes.crt"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547664   10762 flags.go:33] FLAG: --requestheader-extra-headers-prefix="[X-Remote-Extra-]"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547679   10762 flags.go:33] FLAG: --requestheader-group-headers="[X-Remote-Group]"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547702   10762 flags.go:33] FLAG: --requestheader-username-headers="[X-Remote-User]"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547715   10762 flags.go:33] FLAG: --runtime-config=""
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547731   10762 flags.go:33] FLAG: --secure-port="443"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547740   10762 flags.go:33] FLAG: --service-account-api-audiences="[]"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547756   10762 flags.go:33] FLAG: --service-account-issuer=""
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547765   10762 flags.go:33] FLAG: --service-account-key-file="[]"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547790   10762 flags.go:33] FLAG: --service-account-lookup="true"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547799   10762 flags.go:33] FLAG: --service-account-max-token-expiration="0s"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547809   10762 flags.go:33] FLAG: --service-account-signing-key-file=""
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547819   10762 flags.go:33] FLAG: --service-cluster-ip-range="10.254.0.0/16"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547837   10762 flags.go:33] FLAG: --service-node-port-range="30000-32767"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547854   10762 flags.go:33] FLAG: --skip-headers="false"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547866   10762 flags.go:33] FLAG: --ssh-keyfile=""
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547877   10762 flags.go:33] FLAG: --ssh-user=""
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547899   10762 flags.go:33] FLAG: --stderrthreshold="2"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547912   10762 flags.go:33] FLAG: --storage-backend=""
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547921   10762 flags.go:33] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547930   10762 flags.go:33] FLAG: --storage-driver-db="cadvisor"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547939   10762 flags.go:33] FLAG: --storage-driver-host="localhost:8086"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547949   10762 flags.go:33] FLAG: --storage-driver-password="root"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547961   10762 flags.go:33] FLAG: --storage-driver-secure="false"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547971   10762 flags.go:33] FLAG: --storage-driver-table="stats"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547981   10762 flags.go:33] FLAG: --storage-driver-user="root"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.547992   10762 flags.go:33] FLAG: --storage-media-type="application/vnd.kubernetes.protobuf"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.548004   10762 flags.go:33] FLAG: --storage-versions="admission.k8s.io/v1beta1,admissionregistration.k8s.io/v1beta1,apps/v1,auditregistration.k8s.io/v1alpha1,authentication.k8s.io/v1,authorization.k8s.io/v1,autoscaling/v1,batch/v1,certificates.k8s.io/v1beta1,coordination.k8s.io/v1beta1,events.k8s.io/v1beta1,extensions/v1beta1,imagepolicy.k8s.io/v1alpha1,networking.k8s.io/v1,policy/v1beta1,rbac.authorization.k8s.io/v1,scheduling.k8s.io/v1beta1,settings.k8s.io/v1alpha1,storage.k8s.io/v1,v1"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.548118   10762 flags.go:33] FLAG: --target-ram-mb="0"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.548134   10762 flags.go:33] FLAG: --tls-cert-file="/etc/kubernetes/ssl/kubernetes-server.crt"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.548144   10762 flags.go:33] FLAG: --tls-cipher-suites="[]"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.548160   10762 flags.go:33] FLAG: --tls-min-version=""
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.548169   10762 flags.go:33] FLAG: --tls-private-key-file="/etc/kubernetes/ssl/kubernetes-server.key"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.548180   10762 flags.go:33] FLAG: --tls-sni-cert-key="[]"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.548193   10762 flags.go:33] FLAG: --token-auth-file="/srv/kubernetes/known_tokens.csv"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.548202   10762 flags.go:33] FLAG: --v="2"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.548213   10762 flags.go:33] FLAG: --version="false"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.548229   10762 flags.go:33] FLAG: --vmodule=""
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.548240   10762 flags.go:33] FLAG: --watch-cache="true"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.548251   10762 flags.go:33] FLAG: --watch-cache-sizes="[]"
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.548283   10762 server.go:557] external host was not specified, using 172.16.10.36
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.548608   10762 server.go:600] Initializing cache sizes based on 0MB limit
Oct 26 14:00:22 ctl01 kube-apiserver[10762]: I1026 14:00:22.548757   10762 server.go:146] Version: v1.13.5-3+98374c02d2d8c1
Oct 26 14:00:22 ctl01 kubelet[10393]: E1026 14:00:22.685051   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:22 ctl01 kubelet[10393]: E1026 14:00:22.785220   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:22 ctl01 kubelet[10393]: E1026 14:00:22.885502   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:22 ctl01 kubelet[10393]: E1026 14:00:22.985660   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:23 ctl01 kubelet[10393]: E1026 14:00:23.085844   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:23 ctl01 kubelet[10393]: E1026 14:00:23.186040   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:23 ctl01 kubelet[10393]: E1026 14:00:23.286361   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:23 ctl01 kubelet[10393]: E1026 14:00:23.386584   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:23 ctl01 kubelet[10393]: E1026 14:00:23.486860   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:23 ctl01 kubelet[10393]: E1026 14:00:23.587074   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:23 ctl01 kube-apiserver[10762]: I1026 14:00:23.672231   10762 plugins.go:158] Loaded 8 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,MutatingAdmissionWebhook.
Oct 26 14:00:23 ctl01 kube-apiserver[10762]: I1026 14:00:23.672286   10762 plugins.go:161] Loaded 6 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,Priority,PersistentVolumeClaimResize,ValidatingAdmissionWebhook,ResourceQuota.
Oct 26 14:00:23 ctl01 kube-apiserver[10762]: I1026 14:00:23.673074   10762 plugins.go:158] Loaded 8 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,MutatingAdmissionWebhook.
Oct 26 14:00:23 ctl01 kube-apiserver[10762]: I1026 14:00:23.673093   10762 plugins.go:161] Loaded 6 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,Priority,PersistentVolumeClaimResize,ValidatingAdmissionWebhook,ResourceQuota.
Oct 26 14:00:23 ctl01 kubelet[10393]: E1026 14:00:23.687278   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:23 ctl01 kube-apiserver[10762]: I1026 14:00:23.729582   10762 store.go:1414] Monitoring customresourcedefinitions.apiextensions.k8s.io count at <storage-prefix>//apiextensions.k8s.io/customresourcedefinitions
Oct 26 14:00:23 ctl01 kube-apiserver[10762]: I1026 14:00:23.755292   10762 master.go:228] Using reconciler: lease
Oct 26 14:00:23 ctl01 kubelet[10393]: E1026 14:00:23.787468   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:23 ctl01 kube-apiserver[10762]: I1026 14:00:23.862047   10762 store.go:1414] Monitoring podtemplates count at <storage-prefix>//podtemplates
Oct 26 14:00:23 ctl01 kubelet[10393]: E1026 14:00:23.887628   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:23 ctl01 kube-apiserver[10762]: I1026 14:00:23.912429   10762 store.go:1414] Monitoring events count at <storage-prefix>//events
Oct 26 14:00:23 ctl01 kube-apiserver[10762]: I1026 14:00:23.958217   10762 store.go:1414] Monitoring limitranges count at <storage-prefix>//limitranges
Oct 26 14:00:23 ctl01 kubelet[10393]: E1026 14:00:23.987803   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:24 ctl01 kube-apiserver[10762]: I1026 14:00:24.010109   10762 store.go:1414] Monitoring resourcequotas count at <storage-prefix>//resourcequotas
Oct 26 14:00:24 ctl01 kube-apiserver[10762]: I1026 14:00:24.063027   10762 store.go:1414] Monitoring secrets count at <storage-prefix>//secrets
Oct 26 14:00:24 ctl01 kubelet[10393]: E1026 14:00:24.088109   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:24 ctl01 kube-apiserver[10762]: I1026 14:00:24.122751   10762 store.go:1414] Monitoring persistentvolumes count at <storage-prefix>//persistentvolumes
Oct 26 14:00:24 ctl01 kube-apiserver[10762]: I1026 14:00:24.173928   10762 store.go:1414] Monitoring persistentvolumeclaims count at <storage-prefix>//persistentvolumeclaims
Oct 26 14:00:24 ctl01 kubelet[10393]: E1026 14:00:24.188273   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:24 ctl01 kube-apiserver[10762]: I1026 14:00:24.224083   10762 store.go:1414] Monitoring configmaps count at <storage-prefix>//configmaps
Oct 26 14:00:24 ctl01 kube-apiserver[10762]: I1026 14:00:24.275311   10762 store.go:1414] Monitoring namespaces count at <storage-prefix>//namespaces
Oct 26 14:00:24 ctl01 kubelet[10393]: E1026 14:00:24.288423   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:24 ctl01 kube-apiserver[10762]: I1026 14:00:24.331632   10762 store.go:1414] Monitoring endpoints count at <storage-prefix>//services/endpoints
Oct 26 14:00:24 ctl01 kube-apiserver[10762]: I1026 14:00:24.382924   10762 store.go:1414] Monitoring nodes count at <storage-prefix>//minions
Oct 26 14:00:24 ctl01 kubelet[10393]: E1026 14:00:24.388615   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:24 ctl01 kube-apiserver[10762]: I1026 14:00:24.437394   10762 store.go:1414] Monitoring pods count at <storage-prefix>//pods
Oct 26 14:00:24 ctl01 kubelet[10393]: E1026 14:00:24.488809   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:24 ctl01 kube-apiserver[10762]: I1026 14:00:24.499527   10762 store.go:1414] Monitoring serviceaccounts count at <storage-prefix>//serviceaccounts
Oct 26 14:00:24 ctl01 kube-apiserver[10762]: I1026 14:00:24.549948   10762 store.go:1414] Monitoring services count at <storage-prefix>//services/specs
Oct 26 14:00:24 ctl01 kubelet[10393]: E1026 14:00:24.589007   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:24 ctl01 kubelet[10393]: E1026 14:00:24.689215   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:24 ctl01 kube-apiserver[10762]: I1026 14:00:24.699308   10762 store.go:1414] Monitoring replicationcontrollers count at <storage-prefix>//controllers
Oct 26 14:00:24 ctl01 kube-apiserver[10762]: I1026 14:00:24.785462   10762 master.go:407] Skipping disabled API group "auditregistration.k8s.io".
Oct 26 14:00:24 ctl01 kube-apiserver[10762]: I1026 14:00:24.785529   10762 master.go:415] Enabling API group "authentication.k8s.io".
Oct 26 14:00:24 ctl01 kube-apiserver[10762]: I1026 14:00:24.785549   10762 master.go:415] Enabling API group "authorization.k8s.io".
Oct 26 14:00:24 ctl01 kubelet[10393]: E1026 14:00:24.789434   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:24 ctl01 kube-apiserver[10762]: I1026 14:00:24.845754   10762 store.go:1414] Monitoring horizontalpodautoscalers.autoscaling count at <storage-prefix>//horizontalpodautoscalers
Oct 26 14:00:24 ctl01 kubelet[10393]: E1026 14:00:24.889619   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:24 ctl01 kube-apiserver[10762]: I1026 14:00:24.901195   10762 store.go:1414] Monitoring horizontalpodautoscalers.autoscaling count at <storage-prefix>//horizontalpodautoscalers
Oct 26 14:00:24 ctl01 kube-apiserver[10762]: I1026 14:00:24.950372   10762 store.go:1414] Monitoring horizontalpodautoscalers.autoscaling count at <storage-prefix>//horizontalpodautoscalers
Oct 26 14:00:24 ctl01 kube-apiserver[10762]: I1026 14:00:24.950410   10762 master.go:415] Enabling API group "autoscaling".
Oct 26 14:00:24 ctl01 kubelet[10393]: E1026 14:00:24.989914   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:25 ctl01 kube-apiserver[10762]: I1026 14:00:25.016958   10762 store.go:1414] Monitoring jobs.batch count at <storage-prefix>//jobs
Oct 26 14:00:25 ctl01 kube-apiserver[10762]: I1026 14:00:25.071120   10762 store.go:1414] Monitoring cronjobs.batch count at <storage-prefix>//cronjobs
Oct 26 14:00:25 ctl01 kube-apiserver[10762]: I1026 14:00:25.071151   10762 master.go:415] Enabling API group "batch".
Oct 26 14:00:25 ctl01 kubelet[10393]: E1026 14:00:25.090281   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:25 ctl01 kube-apiserver[10762]: I1026 14:00:25.142200   10762 store.go:1414] Monitoring certificatesigningrequests.certificates.k8s.io count at <storage-prefix>//certificatesigningrequests
Oct 26 14:00:25 ctl01 kube-apiserver[10762]: I1026 14:00:25.142242   10762 master.go:415] Enabling API group "certificates.k8s.io".
Oct 26 14:00:25 ctl01 kubelet[10393]: E1026 14:00:25.190556   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:25 ctl01 kube-apiserver[10762]: I1026 14:00:25.199885   10762 store.go:1414] Monitoring leases.coordination.k8s.io count at <storage-prefix>//leases
Oct 26 14:00:25 ctl01 kube-apiserver[10762]: I1026 14:00:25.199913   10762 master.go:415] Enabling API group "coordination.k8s.io".
Oct 26 14:00:25 ctl01 kube-apiserver[10762]: I1026 14:00:25.253806   10762 store.go:1414] Monitoring replicationcontrollers count at <storage-prefix>//controllers
Oct 26 14:00:25 ctl01 kubelet[10393]: E1026 14:00:25.290777   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:25 ctl01 kube-apiserver[10762]: I1026 14:00:25.305550   10762 store.go:1414] Monitoring daemonsets.apps count at <storage-prefix>//daemonsets
Oct 26 14:00:25 ctl01 kube-apiserver[10762]: I1026 14:00:25.357695   10762 store.go:1414] Monitoring deployments.apps count at <storage-prefix>//deployments
Oct 26 14:00:25 ctl01 kubelet[10393]: E1026 14:00:25.391028   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:25 ctl01 kube-apiserver[10762]: I1026 14:00:25.407708   10762 store.go:1414] Monitoring ingresses.extensions count at <storage-prefix>//ingress
Oct 26 14:00:25 ctl01 kube-apiserver[10762]: I1026 14:00:25.462890   10762 store.go:1414] Monitoring podsecuritypolicies.policy count at <storage-prefix>//podsecuritypolicy
Oct 26 14:00:25 ctl01 kubelet[10393]: E1026 14:00:25.491262   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:25 ctl01 kube-apiserver[10762]: I1026 14:00:25.516531   10762 store.go:1414] Monitoring replicasets.apps count at <storage-prefix>//replicasets
Oct 26 14:00:25 ctl01 kube-apiserver[10762]: I1026 14:00:25.572039   10762 store.go:1414] Monitoring networkpolicies.networking.k8s.io count at <storage-prefix>//networkpolicies
Oct 26 14:00:25 ctl01 kube-apiserver[10762]: I1026 14:00:25.572067   10762 master.go:415] Enabling API group "extensions".
Oct 26 14:00:25 ctl01 kubelet[10393]: E1026 14:00:25.591421   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:25 ctl01 kube-apiserver[10762]: I1026 14:00:25.619407   10762 store.go:1414] Monitoring networkpolicies.networking.k8s.io count at <storage-prefix>//networkpolicies
Oct 26 14:00:25 ctl01 kube-apiserver[10762]: I1026 14:00:25.619438   10762 master.go:415] Enabling API group "networking.k8s.io".
Oct 26 14:00:25 ctl01 kube-apiserver[10762]: I1026 14:00:25.675665   10762 store.go:1414] Monitoring poddisruptionbudgets.policy count at <storage-prefix>//poddisruptionbudgets
Oct 26 14:00:25 ctl01 kubelet[10393]: E1026 14:00:25.691557   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:25 ctl01 kube-apiserver[10762]: I1026 14:00:25.728768   10762 store.go:1414] Monitoring podsecuritypolicies.policy count at <storage-prefix>//podsecuritypolicy
Oct 26 14:00:25 ctl01 kube-apiserver[10762]: I1026 14:00:25.728791   10762 master.go:415] Enabling API group "policy".
Oct 26 14:00:25 ctl01 kube-apiserver[10762]: I1026 14:00:25.782445   10762 store.go:1414] Monitoring roles.rbac.authorization.k8s.io count at <storage-prefix>//roles
Oct 26 14:00:25 ctl01 kubelet[10393]: E1026 14:00:25.791764   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:25 ctl01 kube-apiserver[10762]: I1026 14:00:25.840944   10762 store.go:1414] Monitoring rolebindings.rbac.authorization.k8s.io count at <storage-prefix>//rolebindings
Oct 26 14:00:25 ctl01 kubelet[10393]: E1026 14:00:25.891969   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:25 ctl01 kube-apiserver[10762]: I1026 14:00:25.896703   10762 store.go:1414] Monitoring clusterroles.rbac.authorization.k8s.io count at <storage-prefix>//clusterroles
Oct 26 14:00:25 ctl01 kube-apiserver[10762]: I1026 14:00:25.947584   10762 store.go:1414] Monitoring clusterrolebindings.rbac.authorization.k8s.io count at <storage-prefix>//clusterrolebindings
Oct 26 14:00:25 ctl01 kubelet[10393]: E1026 14:00:25.992166   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:26 ctl01 kube-apiserver[10762]: I1026 14:00:25.999557   10762 store.go:1414] Monitoring roles.rbac.authorization.k8s.io count at <storage-prefix>//roles
Oct 26 14:00:26 ctl01 kube-apiserver[10762]: I1026 14:00:26.052057   10762 store.go:1414] Monitoring rolebindings.rbac.authorization.k8s.io count at <storage-prefix>//rolebindings
Oct 26 14:00:26 ctl01 kubelet[10393]: E1026 14:00:26.092455   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:26 ctl01 kube-apiserver[10762]: I1026 14:00:26.104926   10762 store.go:1414] Monitoring clusterroles.rbac.authorization.k8s.io count at <storage-prefix>//clusterroles
Oct 26 14:00:26 ctl01 kube-apiserver[10762]: I1026 14:00:26.160261   10762 store.go:1414] Monitoring clusterrolebindings.rbac.authorization.k8s.io count at <storage-prefix>//clusterrolebindings
Oct 26 14:00:26 ctl01 kube-apiserver[10762]: I1026 14:00:26.160313   10762 master.go:415] Enabling API group "rbac.authorization.k8s.io".
Oct 26 14:00:26 ctl01 kubelet[10393]: E1026 14:00:26.192629   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:26 ctl01 kube-apiserver[10762]: I1026 14:00:26.223171   10762 store.go:1414] Monitoring priorityclasses.scheduling.k8s.io count at <storage-prefix>//priorityclasses
Oct 26 14:00:26 ctl01 kube-apiserver[10762]: I1026 14:00:26.223221   10762 master.go:415] Enabling API group "scheduling.k8s.io".
Oct 26 14:00:26 ctl01 kube-apiserver[10762]: I1026 14:00:26.223321   10762 master.go:407] Skipping disabled API group "settings.k8s.io".
Oct 26 14:00:26 ctl01 kube-apiserver[10762]: I1026 14:00:26.278154   10762 store.go:1414] Monitoring storageclasses.storage.k8s.io count at <storage-prefix>//storageclasses
Oct 26 14:00:26 ctl01 kubelet[10393]: E1026 14:00:26.292913   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:26 ctl01 kube-apiserver[10762]: I1026 14:00:26.336341   10762 store.go:1414] Monitoring volumeattachments.storage.k8s.io count at <storage-prefix>//volumeattachments
Oct 26 14:00:26 ctl01 kube-apiserver[10762]: I1026 14:00:26.391072   10762 store.go:1414] Monitoring storageclasses.storage.k8s.io count at <storage-prefix>//storageclasses
Oct 26 14:00:26 ctl01 kubelet[10393]: E1026 14:00:26.393107   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:26 ctl01 kube-apiserver[10762]: I1026 14:00:26.445272   10762 store.go:1414] Monitoring volumeattachments.storage.k8s.io count at <storage-prefix>//volumeattachments
Oct 26 14:00:26 ctl01 kube-apiserver[10762]: I1026 14:00:26.445301   10762 master.go:415] Enabling API group "storage.k8s.io".
Oct 26 14:00:26 ctl01 kubelet[10393]: E1026 14:00:26.493308   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:26 ctl01 kube-apiserver[10762]: I1026 14:00:26.503317   10762 store.go:1414] Monitoring deployments.apps count at <storage-prefix>//deployments
Oct 26 14:00:26 ctl01 kube-apiserver[10762]: I1026 14:00:26.555290   10762 store.go:1414] Monitoring statefulsets.apps count at <storage-prefix>//statefulsets
Oct 26 14:00:26 ctl01 kubelet[10393]: E1026 14:00:26.593578   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:26 ctl01 kube-apiserver[10762]: I1026 14:00:26.606040   10762 store.go:1414] Monitoring controllerrevisions.apps count at <storage-prefix>//controllerrevisions
Oct 26 14:00:26 ctl01 kube-apiserver[10762]: I1026 14:00:26.654269   10762 store.go:1414] Monitoring deployments.apps count at <storage-prefix>//deployments
Oct 26 14:00:26 ctl01 kubelet[10393]: E1026 14:00:26.693784   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:26 ctl01 kube-apiserver[10762]: I1026 14:00:26.702725   10762 store.go:1414] Monitoring statefulsets.apps count at <storage-prefix>//statefulsets
Oct 26 14:00:26 ctl01 kube-apiserver[10762]: I1026 14:00:26.759590   10762 store.go:1414] Monitoring daemonsets.apps count at <storage-prefix>//daemonsets
Oct 26 14:00:26 ctl01 kubelet[10393]: E1026 14:00:26.794088   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:26 ctl01 kube-apiserver[10762]: I1026 14:00:26.813496   10762 store.go:1414] Monitoring replicasets.apps count at <storage-prefix>//replicasets
Oct 26 14:00:26 ctl01 kube-apiserver[10762]: I1026 14:00:26.874666   10762 store.go:1414] Monitoring controllerrevisions.apps count at <storage-prefix>//controllerrevisions
Oct 26 14:00:26 ctl01 kubelet[10393]: E1026 14:00:26.894328   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:26 ctl01 kube-apiserver[10762]: I1026 14:00:26.928731   10762 store.go:1414] Monitoring deployments.apps count at <storage-prefix>//deployments
Oct 26 14:00:26 ctl01 kube-apiserver[10762]: I1026 14:00:26.981495   10762 store.go:1414] Monitoring statefulsets.apps count at <storage-prefix>//statefulsets
Oct 26 14:00:26 ctl01 kubelet[10393]: E1026 14:00:26.994528   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:27 ctl01 kube-apiserver[10762]: I1026 14:00:27.032919   10762 store.go:1414] Monitoring daemonsets.apps count at <storage-prefix>//daemonsets
Oct 26 14:00:27 ctl01 kube-apiserver[10762]: I1026 14:00:27.089730   10762 store.go:1414] Monitoring replicasets.apps count at <storage-prefix>//replicasets
Oct 26 14:00:27 ctl01 kubelet[10393]: E1026 14:00:27.094846   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:27 ctl01 kube-apiserver[10762]: I1026 14:00:27.143356   10762 store.go:1414] Monitoring controllerrevisions.apps count at <storage-prefix>//controllerrevisions
Oct 26 14:00:27 ctl01 kube-apiserver[10762]: I1026 14:00:27.143386   10762 master.go:415] Enabling API group "apps".
Oct 26 14:00:27 ctl01 kube-apiserver[10762]: I1026 14:00:27.194432   10762 store.go:1414] Monitoring validatingwebhookconfigurations.admissionregistration.k8s.io count at <storage-prefix>//validatingwebhookconfigurations
Oct 26 14:00:27 ctl01 kubelet[10393]: E1026 14:00:27.195039   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:27 ctl01 kube-apiserver[10762]: I1026 14:00:27.247043   10762 store.go:1414] Monitoring mutatingwebhookconfigurations.admissionregistration.k8s.io count at <storage-prefix>//mutatingwebhookconfigurations
Oct 26 14:00:27 ctl01 kube-apiserver[10762]: I1026 14:00:27.247139   10762 master.go:415] Enabling API group "admissionregistration.k8s.io".
Oct 26 14:00:27 ctl01 kubelet[10393]: E1026 14:00:27.295189   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:27 ctl01 kube-apiserver[10762]: I1026 14:00:27.300799   10762 store.go:1414] Monitoring events count at <storage-prefix>//events
Oct 26 14:00:27 ctl01 kube-apiserver[10762]: I1026 14:00:27.301198   10762 master.go:415] Enabling API group "events.k8s.io".
Oct 26 14:00:27 ctl01 kubelet[10393]: E1026 14:00:27.395356   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:27 ctl01 kube-apiserver[10762]: W1026 14:00:27.430699   10762 genericapiserver.go:338] Skipping API batch/v2alpha1 because it has no resources.
Oct 26 14:00:27 ctl01 kubelet[10393]: E1026 14:00:27.495595   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:27 ctl01 kubelet[10393]: E1026 14:00:27.595891   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:27 ctl01 kube-apiserver[10762]: W1026 14:00:27.644791   10762 genericapiserver.go:338] Skipping API rbac.authorization.k8s.io/v1alpha1 because it has no resources.
Oct 26 14:00:27 ctl01 kube-apiserver[10762]: W1026 14:00:27.670913   10762 genericapiserver.go:338] Skipping API scheduling.k8s.io/v1alpha1 because it has no resources.
Oct 26 14:00:27 ctl01 kubelet[10393]: E1026 14:00:27.696111   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:27 ctl01 kube-apiserver[10762]: W1026 14:00:27.706473   10762 genericapiserver.go:338] Skipping API storage.k8s.io/v1alpha1 because it has no resources.
Oct 26 14:00:27 ctl01 kubelet[10393]: E1026 14:00:27.796487   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:27 ctl01 kube-apiserver[10762]: W1026 14:00:27.875536   10762 genericapiserver.go:338] Skipping API admissionregistration.k8s.io/v1alpha1 because it has no resources.
Oct 26 14:00:27 ctl01 kubelet[10393]: E1026 14:00:27.896801   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:27 ctl01 kube-apiserver[10762]: [restful] 2019/10/26 14:00:27 log.go:33: [restful/swagger] listing is available at https://172.16.10.36:443/swaggerapi
Oct 26 14:00:27 ctl01 kube-apiserver[10762]: [restful] 2019/10/26 14:00:27 log.go:33: [restful/swagger] https://172.16.10.36:443/swaggerui/ is mapped to folder /swagger-ui/
Oct 26 14:00:27 ctl01 kubelet[10393]: E1026 14:00:27.997054   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:28 ctl01 kubelet[10393]: E1026 14:00:28.097413   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:28 ctl01 kubelet[10393]: E1026 14:00:28.197629   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:28 ctl01 kubelet[10393]: E1026 14:00:28.297907   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:28 ctl01 kubelet[10393]: E1026 14:00:28.398159   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:28 ctl01 kubelet[10393]: E1026 14:00:28.498450   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:28 ctl01 kubelet[10393]: E1026 14:00:28.599804   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:28 ctl01 kubelet[10393]: E1026 14:00:28.700725   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:28 ctl01 kubelet[10393]: E1026 14:00:28.801168   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:28 ctl01 kubelet[10393]: E1026 14:00:28.902007   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:29 ctl01 kubelet[10393]: E1026 14:00:29.002278   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:29 ctl01 kubelet[10393]: E1026 14:00:29.102588   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:29 ctl01 kubelet[10393]: E1026 14:00:29.202900   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:29 ctl01 kubelet[10393]: E1026 14:00:29.303143   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:29 ctl01 kubelet[10393]: E1026 14:00:29.403403   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:29 ctl01 kubelet[10393]: E1026 14:00:29.503624   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:29 ctl01 kubelet[10393]: E1026 14:00:29.603933   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:29 ctl01 kubelet[10393]: E1026 14:00:29.630794   10393 eviction_manager.go:247] eviction manager: failed to get summary stats: failed to get node info: node "ctl01" not found
Oct 26 14:00:29 ctl01 kubelet[10393]: E1026 14:00:29.704330   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:29 ctl01 kubelet[10393]: E1026 14:00:29.804612   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:29 ctl01 kubelet[10393]: E1026 14:00:29.905773   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:30 ctl01 kubelet[10393]: E1026 14:00:30.006007   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:30 ctl01 kubelet[10393]: E1026 14:00:30.106190   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:30 ctl01 kubelet[10393]: E1026 14:00:30.206368   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:30 ctl01 kubelet[10393]: E1026 14:00:30.306637   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:30 ctl01 kubelet[10393]: E1026 14:00:30.406901   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:30 ctl01 kube-apiserver[10762]: [restful] 2019/10/26 14:00:30 log.go:33: [restful/swagger] listing is available at https://172.16.10.36:443/swaggerapi
Oct 26 14:00:30 ctl01 kube-apiserver[10762]: [restful] 2019/10/26 14:00:30 log.go:33: [restful/swagger] https://172.16.10.36:443/swaggerui/ is mapped to folder /swagger-ui/
Oct 26 14:00:30 ctl01 kubelet[10393]: E1026 14:00:30.507125   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:30 ctl01 kube-apiserver[10762]: I1026 14:00:30.551306   10762 plugins.go:158] Loaded 8 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,MutatingAdmissionWebhook.
Oct 26 14:00:30 ctl01 kube-apiserver[10762]: I1026 14:00:30.551349   10762 plugins.go:161] Loaded 6 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,Priority,PersistentVolumeClaimResize,ValidatingAdmissionWebhook,ResourceQuota.
Oct 26 14:00:30 ctl01 kubelet[10393]: E1026 14:00:30.607391   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:30 ctl01 kube-apiserver[10762]: I1026 14:00:30.620894   10762 store.go:1414] Monitoring apiservices.apiregistration.k8s.io count at <storage-prefix>//apiregistration.k8s.io/apiservices
Oct 26 14:00:30 ctl01 kube-apiserver[10762]: I1026 14:00:30.680956   10762 store.go:1414] Monitoring apiservices.apiregistration.k8s.io count at <storage-prefix>//apiregistration.k8s.io/apiservices
Oct 26 14:00:30 ctl01 kubelet[10393]: E1026 14:00:30.707649   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:30 ctl01 kubelet[10393]: E1026 14:00:30.807996   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:30 ctl01 kubelet[10393]: E1026 14:00:30.908267   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:31 ctl01 kubelet[10393]: E1026 14:00:31.008461   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:31 ctl01 kubelet[10393]: E1026 14:00:31.108624   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:31 ctl01 kubelet[10393]: E1026 14:00:31.208786   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:31 ctl01 kubelet[10393]: E1026 14:00:31.308991   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:31 ctl01 kubelet[10393]: E1026 14:00:31.409254   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:31 ctl01 kubelet[10393]: E1026 14:00:31.509570   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:31 ctl01 kubelet[10393]: E1026 14:00:31.609873   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:31 ctl01 kubelet[10393]: E1026 14:00:31.710203   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:31 ctl01 kubelet[10393]: E1026 14:00:31.810470   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:31 ctl01 kubelet[10393]: E1026 14:00:31.910637   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:32 ctl01 kubelet[10393]: E1026 14:00:32.010854   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:32 ctl01 kubelet[10393]: E1026 14:00:32.111017   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:32 ctl01 kubelet[10393]: E1026 14:00:32.211264   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:32 ctl01 kubelet[10393]: E1026 14:00:32.311463   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:32 ctl01 kubelet[10393]: E1026 14:00:32.411692   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:32 ctl01 kubelet[10393]: E1026 14:00:32.511919   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:32 ctl01 kubelet[10393]: E1026 14:00:32.590076   10393 kubelet_node_status.go:94] Unable to register node "ctl01" with API server: Post https://172.16.10.36:443/api/v1/nodes: net/http: TLS handshake timeout
Oct 26 14:00:32 ctl01 kubelet[10393]: E1026 14:00:32.612218   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:32 ctl01 kubelet[10393]: E1026 14:00:32.712605   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:32 ctl01 kubelet[10393]: E1026 14:00:32.812816   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:32 ctl01 kube-proxy[8230]: E1026 14:00:32.855452    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: Get https://172.16.10.36:443/api/v1/endpoints?limit=500&resourceVersion=0: net/http: TLS handshake timeout
Oct 26 14:00:32 ctl01 kube-proxy[8230]: E1026 14:00:32.890347    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: net/http: TLS handshake timeout
Oct 26 14:00:32 ctl01 kubelet[10393]: E1026 14:00:32.913162   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:33 ctl01 kubelet[10393]: E1026 14:00:33.013377   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:33 ctl01 kubelet[10393]: E1026 14:00:33.113581   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:33 ctl01 kubelet[10393]: E1026 14:00:33.213798   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:33 ctl01 kubelet[10393]: E1026 14:00:33.313986   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:33 ctl01 kubelet[10393]: E1026 14:00:33.414228   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:33 ctl01 kubelet[10393]: E1026 14:00:33.475305   10393 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Get https://172.16.10.36:443/api/v1/pods?fieldSelector=spec.nodeName%3Dctl01&limit=500&resourceVersion=0: net/http: TLS handshake timeout
Oct 26 14:00:33 ctl01 kubelet[10393]: E1026 14:00:33.476153   10393 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://172.16.10.36:443/api/v1/nodes?fieldSelector=metadata.name%3Dctl01&limit=500&resourceVersion=0: net/http: TLS handshake timeout
Oct 26 14:00:33 ctl01 kubelet[10393]: E1026 14:00:33.477624   10393 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444: Failed to list *v1.Service: Get https://172.16.10.36:443/api/v1/services?limit=500&resourceVersion=0: net/http: TLS handshake timeout
Oct 26 14:00:33 ctl01 kubelet[10393]: E1026 14:00:33.514458   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:33 ctl01 kubelet[10393]: E1026 14:00:33.614663   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:33 ctl01 kubelet[10393]: E1026 14:00:33.714860   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:33 ctl01 kubelet[10393]: E1026 14:00:33.815043   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:33 ctl01 kubelet[10393]: E1026 14:00:33.915303   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:34 ctl01 kubelet[10393]: E1026 14:00:34.015541   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:34 ctl01 kubelet[10393]: E1026 14:00:34.115745   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:34 ctl01 kubelet[10393]: E1026 14:00:34.215924   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:34 ctl01 kube-proxy[8230]: E1026 14:00:34.296638    8230 event.go:212] Unable to write event: 'Post https://172.16.10.36:443/api/v1/namespaces/default/events: net/http: TLS handshake timeout' (may retry after sleeping)
Oct 26 14:00:34 ctl01 kubelet[10393]: E1026 14:00:34.316169   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:34 ctl01 kubelet[10393]: E1026 14:00:34.416342   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:34 ctl01 kubelet[10393]: E1026 14:00:34.516517   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:34 ctl01 kubelet[10393]: E1026 14:00:34.616735   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:34 ctl01 kubelet[10393]: E1026 14:00:34.716961   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:34 ctl01 kubelet[10393]: E1026 14:00:34.817308   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:34 ctl01 kubelet[10393]: E1026 14:00:34.917594   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:35 ctl01 kubelet[10393]: E1026 14:00:35.017773   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:35 ctl01 kubelet[10393]: E1026 14:00:35.118018   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:35 ctl01 kubelet[10393]: E1026 14:00:35.218227   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:35 ctl01 kubelet[10393]: E1026 14:00:35.318385   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:35 ctl01 kubelet[10393]: E1026 14:00:35.418601   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:35 ctl01 kubelet[10393]: E1026 14:00:35.518892   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:35 ctl01 kubelet[10393]: E1026 14:00:35.619091   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:35 ctl01 kubelet[10393]: E1026 14:00:35.719242   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:35 ctl01 kubelet[10393]: I1026 14:00:35.790358   10393 kubelet_node_status.go:279] Setting node annotation to enable volume controller attach/detach
Oct 26 14:00:35 ctl01 kubelet[10393]: I1026 14:00:35.790693   10393 setters.go:72] Using node IP: "172.16.10.36"
Oct 26 14:00:35 ctl01 kubelet[10393]: I1026 14:00:35.791655   10393 kubelet_node_status.go:447] Recording NodeHasSufficientMemory event message for node ctl01
Oct 26 14:00:35 ctl01 kubelet[10393]: I1026 14:00:35.791690   10393 kubelet_node_status.go:447] Recording NodeHasNoDiskPressure event message for node ctl01
Oct 26 14:00:35 ctl01 kubelet[10393]: I1026 14:00:35.791704   10393 kubelet_node_status.go:447] Recording NodeHasSufficientPID event message for node ctl01
Oct 26 14:00:35 ctl01 kubelet[10393]: I1026 14:00:35.791728   10393 kubelet_node_status.go:72] Attempting to register node ctl01
Oct 26 14:00:35 ctl01 kubelet[10393]: E1026 14:00:35.819394   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:35 ctl01 kube-apiserver[10762]: I1026 14:00:35.883887   10762 deprecated_insecure_serving.go:51] Serving insecurely on [::]:8080
Oct 26 14:00:35 ctl01 kube-apiserver[10762]: I1026 14:00:35.885663   10762 secure_serving.go:116] Serving securely on [::]:443
Oct 26 14:00:35 ctl01 kube-apiserver[10762]: I1026 14:00:35.886080   10762 crd_finalizer.go:242] Starting CRDFinalizer
Oct 26 14:00:35 ctl01 systemd[1]: Started Kubernetes API Server.
Oct 26 14:00:35 ctl01 kube-apiserver[10762]: I1026 14:00:35.886602   10762 available_controller.go:283] Starting AvailableConditionController
Oct 26 14:00:35 ctl01 kube-apiserver[10762]: I1026 14:00:35.886902   10762 cache.go:32] Waiting for caches to sync for AvailableConditionController controller
Oct 26 14:00:35 ctl01 kube-apiserver[10762]: I1026 14:00:35.888488   10762 apiservice_controller.go:90] Starting APIServiceRegistrationController
Oct 26 14:00:35 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-active', 'kube-apiserver.service'] in directory '/root'
Oct 26 14:00:35 ctl01 kube-apiserver[10762]: I1026 14:00:35.888978   10762 cache.go:32] Waiting for caches to sync for APIServiceRegistrationController controller
Oct 26 14:00:35 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-enabled', 'kube-apiserver.service'] in directory '/root'
Oct 26 14:00:35 ctl01 kube-apiserver[10762]: I1026 14:00:35.909738   10762 autoregister_controller.go:136] Starting autoregister controller
Oct 26 14:00:35 ctl01 kube-apiserver[10762]: I1026 14:00:35.909768   10762 cache.go:32] Waiting for caches to sync for autoregister controller
Oct 26 14:00:35 ctl01 kube-apiserver[10762]: I1026 14:00:35.909829   10762 controller.go:84] Starting OpenAPI AggregationController
Oct 26 14:00:35 ctl01 kube-apiserver[10762]: I1026 14:00:35.911113   10762 customresource_discovery_controller.go:203] Starting DiscoveryController
Oct 26 14:00:35 ctl01 kube-apiserver[10762]: I1026 14:00:35.911154   10762 naming_controller.go:284] Starting NamingConditionController
Oct 26 14:00:35 ctl01 kube-apiserver[10762]: I1026 14:00:35.911196   10762 establishing_controller.go:73] Starting EstablishingController
Oct 26 14:00:35 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-enabled', 'kube-apiserver.service'] in directory '/root'
Oct 26 14:00:35 ctl01 kubelet[10393]: E1026 14:00:35.928099   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:35 ctl01 kube-apiserver[10762]: I1026 14:00:35.928598   10762 log.go:172] http: TLS handshake error from 172.16.10.36:47478: EOF
Oct 26 14:00:35 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemd-run', '--scope', 'systemctl', 'enable', 'kube-apiserver.service'] in directory '/root'
Oct 26 14:00:35 ctl01 kube-apiserver[10762]: I1026 14:00:35.953774   10762 log.go:172] http: TLS handshake error from 172.16.10.36:47480: EOF
Oct 26 14:00:35 ctl01 kube-apiserver[10762]: I1026 14:00:35.958633   10762 log.go:172] http: TLS handshake error from 172.16.10.36:47470: EOF
Oct 26 14:00:35 ctl01 kubelet[10393]: I1026 14:00:35.960090   10393 kubelet_node_status.go:75] Successfully registered node ctl01
Oct 26 14:00:35 ctl01 kube-proxy[8230]: E1026 14:00:35.961724    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: services is forbidden: User "system:kube-proxy" cannot list resource "services" in API group "" at the cluster scope
Oct 26 14:00:35 ctl01 systemd[1]: Started /bin/systemctl enable kube-apiserver.service.
Oct 26 14:00:35 ctl01 kube-apiserver[10762]: I1026 14:00:35.969591   10762 log.go:172] http: TLS handshake error from 172.16.10.55:47250: EOF
Oct 26 14:00:35 ctl01 kubelet[10393]: I1026 14:00:35.970887   10393 setters.go:72] Using node IP: "172.16.10.36"
Oct 26 14:00:35 ctl01 systemd[1]: Reloading.
Oct 26 14:00:35 ctl01 kube-apiserver[10762]: I1026 14:00:35.990866   10762 cache.go:39] Caches are synced for APIServiceRegistrationController controller
Oct 26 14:00:35 ctl01 kube-apiserver[10762]: I1026 14:00:35.990927   10762 cache.go:39] Caches are synced for AvailableConditionController controller
Oct 26 14:00:35 ctl01 kube-apiserver[10762]: I1026 14:00:35.999904   10762 log.go:172] http: TLS handshake error from 172.16.10.55:47262: EOF
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.011744   10762 log.go:172] http: TLS handshake error from 172.16.10.55:47252: EOF
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.011886   10762 cache.go:39] Caches are synced for autoregister controller
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.021265   10762 log.go:172] http: TLS handshake error from 172.16.10.36:47506: EOF
Oct 26 14:00:36 ctl01 kubelet[10393]: E1026 14:00:36.028285   10393 kubelet.go:2266] node "ctl01" not found
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.029135   10762 log.go:172] http: TLS handshake error from 172.16.10.55:47254: EOF
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.038353   10762 log.go:172] http: TLS handshake error from 172.16.10.36:47472: EOF
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.040251   10762 cacher.go:605] cacher (*apiregistration.APIService): 1 objects queued in incoming channel.
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.045574   10762 log.go:172] http: TLS handshake error from 172.16.10.36:47474: EOF
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.073370   10762 log.go:172] http: TLS handshake error from 172.16.10.55:47264: EOF
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.082557   10762 log.go:172] http: TLS handshake error from 172.16.10.55:47256: EOF
Oct 26 14:00:36 ctl01 systemd[1]: kubelet.service: Dependency Conflicts=cadvisor.service dropped, merged into kubelet.service
Oct 26 14:00:36 ctl01 systemd[1]: kubelet.service: Dependency ConflictedBy=cadvisor.service dropped, merged into kubelet.service
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.107407   10762 log.go:172] http: TLS handshake error from 172.16.10.56:57278: EOF
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.109053   10762 crdregistration_controller.go:112] Starting crd-autoregister controller
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.109073   10762 controller_utils.go:1027] Waiting for caches to sync for crd-autoregister controller
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.114066   10762 log.go:172] http: TLS handshake error from 172.16.10.56:57286: EOF
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.114946   10762 log.go:172] http: TLS handshake error from 172.16.10.56:57280: EOF
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.116146   10762 log.go:172] http: TLS handshake error from 172.16.10.56:57282: EOF
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.116492   10762 log.go:172] http: TLS handshake error from 172.16.10.36:47476: EOF
Oct 26 14:00:36 ctl01 systemd[1]: Started ACPI event daemon.
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.117894   10762 log.go:172] http: TLS handshake error from 172.16.10.55:47258: EOF
Oct 26 14:00:36 ctl01 kube-proxy[8230]: E1026 14:00:36.118415    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: endpoints is forbidden: User "system:kube-proxy" cannot list resource "endpoints" in API group "" at the cluster scope
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.119369   10762 log.go:172] http: TLS handshake error from 172.16.10.56:57284: EOF
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.120772   10762 log.go:172] http: TLS handshake error from 172.16.10.55:47260: EOF
Oct 26 14:00:36 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-enabled', 'kube-apiserver.service'] in directory '/root'
Oct 26 14:00:36 ctl01 kubelet[10393]: I1026 14:00:36.124448   10393 kubelet.go:1908] SyncLoop (ADD, "api"): ""
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: W1026 14:00:36.135198   10762 lease.go:222] Resetting endpoints for master service "kubernetes" to [172.16.10.36]
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.135889   10762 controller.go:608] quota admission added evaluator for: endpoints
Oct 26 14:00:36 ctl01 salt-minion[4526]: [INFO    ] {'kube-apiserver': True}
Oct 26 14:00:36 ctl01 salt-minion[4526]: [INFO    ] Completed state [kube-apiserver] at time 14:00:36.139061 duration_in_ms=13842.148
Oct 26 14:00:36 ctl01 salt-minion[4526]: [INFO    ] Running state [kube-scheduler] at time 14:00:36.144018
Oct 26 14:00:36 ctl01 salt-minion[4526]: [INFO    ] Executing state service.running for [kube-scheduler]
Oct 26 14:00:36 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'status', 'kube-scheduler.service', '-n', '0'] in directory '/root'
Oct 26 14:00:36 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-active', 'kube-scheduler.service'] in directory '/root'
Oct 26 14:00:36 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-enabled', 'kube-scheduler.service'] in directory '/root'
Oct 26 14:00:36 ctl01 kubelet[10393]: I1026 14:00:36.178912   10393 reconciler.go:154] Reconciler: start to sync state
Oct 26 14:00:36 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemd-run', '--scope', 'systemctl', 'start', 'kube-scheduler.service'] in directory '/root'
Oct 26 14:00:36 ctl01 systemd[1]: Started /bin/systemctl start kube-scheduler.service.
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.209898   10762 controller_utils.go:1034] Caches are synced for crd-autoregister controller
Oct 26 14:00:36 ctl01 systemd[1]: Started Kubernetes Scheduler Plugin.
Oct 26 14:00:36 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-active', 'kube-scheduler.service'] in directory '/root'
Oct 26 14:00:36 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-enabled', 'kube-scheduler.service'] in directory '/root'
Oct 26 14:00:36 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-enabled', 'kube-scheduler.service'] in directory '/root'
Oct 26 14:00:36 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemd-run', '--scope', 'systemctl', 'enable', 'kube-scheduler.service'] in directory '/root'
Oct 26 14:00:36 ctl01 systemd[1]: Started /bin/systemctl enable kube-scheduler.service.
Oct 26 14:00:36 ctl01 systemd[1]: Reloading.
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.375684   10878 flags.go:33] FLAG: --address="0.0.0.0"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.375927   10878 flags.go:33] FLAG: --algorithm-provider=""
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.375948   10878 flags.go:33] FLAG: --alsologtostderr="false"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.375957   10878 flags.go:33] FLAG: --application-metrics-count-limit="100"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.375964   10878 flags.go:33] FLAG: --authentication-kubeconfig=""
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.375970   10878 flags.go:33] FLAG: --authentication-skip-lookup="false"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.375977   10878 flags.go:33] FLAG: --authentication-token-webhook-cache-ttl="10s"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.375985   10878 flags.go:33] FLAG: --authentication-tolerate-lookup-failure="true"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.375991   10878 flags.go:33] FLAG: --authorization-always-allow-paths="[/healthz]"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376004   10878 flags.go:33] FLAG: --authorization-kubeconfig=""
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376010   10878 flags.go:33] FLAG: --authorization-webhook-cache-authorized-ttl="10s"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376016   10878 flags.go:33] FLAG: --authorization-webhook-cache-unauthorized-ttl="10s"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376022   10878 flags.go:33] FLAG: --azure-container-registry-config=""
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376027   10878 flags.go:33] FLAG: --bind-address="0.0.0.0"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376035   10878 flags.go:33] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376041   10878 flags.go:33] FLAG: --cert-dir=""
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376047   10878 flags.go:33] FLAG: --client-ca-file=""
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376052   10878 flags.go:33] FLAG: --cloud-provider-gce-lb-src-cidrs="130.211.0.0/22,209.85.152.0/22,209.85.204.0/22,35.191.0.0/16"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376062   10878 flags.go:33] FLAG: --config=""
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376068   10878 flags.go:33] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376074   10878 flags.go:33] FLAG: --containerd="unix:///var/run/containerd.sock"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376080   10878 flags.go:33] FLAG: --contention-profiling="false"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376086   10878 flags.go:33] FLAG: --default-not-ready-toleration-seconds="300"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376092   10878 flags.go:33] FLAG: --default-unreachable-toleration-seconds="300"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376098   10878 flags.go:33] FLAG: --docker="unix:///var/run/docker.sock"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376104   10878 flags.go:33] FLAG: --docker-env-metadata-whitelist=""
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376110   10878 flags.go:33] FLAG: --docker-only="false"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376115   10878 flags.go:33] FLAG: --docker-root="/var/lib/docker"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376122   10878 flags.go:33] FLAG: --docker-tls="false"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376127   10878 flags.go:33] FLAG: --docker-tls-ca="ca.pem"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376133   10878 flags.go:33] FLAG: --docker-tls-cert="cert.pem"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376138   10878 flags.go:33] FLAG: --docker-tls-key="key.pem"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376144   10878 flags.go:33] FLAG: --enable-load-reader="false"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376150   10878 flags.go:33] FLAG: --event-storage-age-limit="default=0"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376164   10878 flags.go:33] FLAG: --event-storage-event-limit="default=0"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376170   10878 flags.go:33] FLAG: --failure-domains="kubernetes.io/hostname,failure-domain.beta.kubernetes.io/zone,failure-domain.beta.kubernetes.io/region"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376179   10878 flags.go:33] FLAG: --feature-gates=""
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376188   10878 flags.go:33] FLAG: --global-housekeeping-interval="1m0s"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376195   10878 flags.go:33] FLAG: --hard-pod-affinity-symmetric-weight="1"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376202   10878 flags.go:33] FLAG: --help="false"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376208   10878 flags.go:33] FLAG: --housekeeping-interval="10s"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376214   10878 flags.go:33] FLAG: --http2-max-streams-per-connection="0"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376221   10878 flags.go:33] FLAG: --kube-api-burst="100"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376228   10878 flags.go:33] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376234   10878 flags.go:33] FLAG: --kube-api-qps="50"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376242   10878 flags.go:33] FLAG: --kubeconfig="/etc/kubernetes/scheduler.kubeconfig"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376248   10878 flags.go:33] FLAG: --leader-elect="true"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376254   10878 flags.go:33] FLAG: --leader-elect-lease-duration="15s"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376260   10878 flags.go:33] FLAG: --leader-elect-renew-deadline="10s"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376269   10878 flags.go:33] FLAG: --leader-elect-resource-lock="endpoints"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376275   10878 flags.go:33] FLAG: --leader-elect-retry-period="2s"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376280   10878 flags.go:33] FLAG: --lock-object-name="kube-scheduler"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376286   10878 flags.go:33] FLAG: --lock-object-namespace="kube-system"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376292   10878 flags.go:33] FLAG: --log-backtrace-at=":0"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376301   10878 flags.go:33] FLAG: --log-cadvisor-usage="false"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376307   10878 flags.go:33] FLAG: --log-dir=""
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376312   10878 flags.go:33] FLAG: --log-file=""
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376318   10878 flags.go:33] FLAG: --log-flush-frequency="5s"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376323   10878 flags.go:33] FLAG: --logtostderr="true"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376329   10878 flags.go:33] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376336   10878 flags.go:33] FLAG: --master=""
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376341   10878 flags.go:33] FLAG: --mesos-agent="127.0.0.1:5051"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376347   10878 flags.go:33] FLAG: --mesos-agent-timeout="10s"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376353   10878 flags.go:33] FLAG: --policy-config-file=""
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376358   10878 flags.go:33] FLAG: --policy-configmap=""
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376363   10878 flags.go:33] FLAG: --policy-configmap-namespace="kube-system"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376378   10878 flags.go:33] FLAG: --port="10251"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376384   10878 flags.go:33] FLAG: --profiling="false"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376390   10878 flags.go:33] FLAG: --requestheader-allowed-names="[]"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376403   10878 flags.go:33] FLAG: --requestheader-client-ca-file=""
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376409   10878 flags.go:33] FLAG: --requestheader-extra-headers-prefix="[x-remote-extra-]"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376419   10878 flags.go:33] FLAG: --requestheader-group-headers="[x-remote-group]"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376429   10878 flags.go:33] FLAG: --requestheader-username-headers="[x-remote-user]"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376439   10878 flags.go:33] FLAG: --scheduler-name="default-scheduler"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376445   10878 flags.go:33] FLAG: --secure-port="10259"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376451   10878 flags.go:33] FLAG: --skip-headers="false"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376457   10878 flags.go:33] FLAG: --stderrthreshold="2"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376462   10878 flags.go:33] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376468   10878 flags.go:33] FLAG: --storage-driver-db="cadvisor"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376474   10878 flags.go:33] FLAG: --storage-driver-host="localhost:8086"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376480   10878 flags.go:33] FLAG: --storage-driver-password="root"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376486   10878 flags.go:33] FLAG: --storage-driver-secure="false"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376491   10878 flags.go:33] FLAG: --storage-driver-table="stats"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376497   10878 flags.go:33] FLAG: --storage-driver-user="root"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376502   10878 flags.go:33] FLAG: --tls-cert-file=""
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376508   10878 flags.go:33] FLAG: --tls-cipher-suites="[]"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376518   10878 flags.go:33] FLAG: --tls-min-version=""
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376524   10878 flags.go:33] FLAG: --tls-private-key-file=""
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376529   10878 flags.go:33] FLAG: --tls-sni-cert-key="[]"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376537   10878 flags.go:33] FLAG: --use-legacy-policy-config="false"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376543   10878 flags.go:33] FLAG: --v="2"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376549   10878 flags.go:33] FLAG: --version="false"
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376557   10878 flags.go:33] FLAG: --vmodule=""
Oct 26 14:00:36 ctl01 kube-scheduler[10878]: I1026 14:00:36.376563   10878 flags.go:33] FLAG: --write-config-to=""
Oct 26 14:00:36 ctl01 systemd[1]: kubelet.service: Dependency Conflicts=cadvisor.service dropped, merged into kubelet.service
Oct 26 14:00:36 ctl01 systemd[1]: kubelet.service: Dependency ConflictedBy=cadvisor.service dropped, merged into kubelet.service
Oct 26 14:00:36 ctl01 systemd[1]: Started ACPI event daemon.
Oct 26 14:00:36 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-enabled', 'kube-scheduler.service'] in directory '/root'
Oct 26 14:00:36 ctl01 salt-minion[4526]: [INFO    ] {'kube-scheduler': True}
Oct 26 14:00:36 ctl01 salt-minion[4526]: [INFO    ] Completed state [kube-scheduler] at time 14:00:36.444648 duration_in_ms=300.631
Oct 26 14:00:36 ctl01 salt-minion[4526]: [INFO    ] Running state [kube-controller-manager] at time 14:00:36.449542
Oct 26 14:00:36 ctl01 salt-minion[4526]: [INFO    ] Executing state service.running for [kube-controller-manager]
Oct 26 14:00:36 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'status', 'kube-controller-manager.service', '-n', '0'] in directory '/root'
Oct 26 14:00:36 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-active', 'kube-controller-manager.service'] in directory '/root'
Oct 26 14:00:36 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-enabled', 'kube-controller-manager.service'] in directory '/root'
Oct 26 14:00:36 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemd-run', '--scope', 'systemctl', 'start', 'kube-controller-manager.service'] in directory '/root'
Oct 26 14:00:36 ctl01 systemd[1]: Started /bin/systemctl start kube-controller-manager.service.
Oct 26 14:00:36 ctl01 systemd[1]: Started Kubernetes Controller Manager.
Oct 26 14:00:36 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-active', 'kube-controller-manager.service'] in directory '/root'
Oct 26 14:00:36 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-enabled', 'kube-controller-manager.service'] in directory '/root'
Oct 26 14:00:36 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-enabled', 'kube-controller-manager.service'] in directory '/root'
Oct 26 14:00:36 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemd-run', '--scope', 'systemctl', 'enable', 'kube-controller-manager.service'] in directory '/root'
Oct 26 14:00:36 ctl01 systemd[1]: Started /bin/systemctl enable kube-controller-manager.service.
Oct 26 14:00:36 ctl01 systemd[1]: Reloading.
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.696730   10954 flags.go:33] FLAG: --address="0.0.0.0"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697537   10954 flags.go:33] FLAG: --allocate-node-cidrs="false"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697559   10954 flags.go:33] FLAG: --allow-untagged-cloud="false"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697566   10954 flags.go:33] FLAG: --alsologtostderr="false"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697574   10954 flags.go:33] FLAG: --application-metrics-count-limit="100"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697581   10954 flags.go:33] FLAG: --attach-detach-reconcile-sync-period="1m0s"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697588   10954 flags.go:33] FLAG: --authentication-kubeconfig=""
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697595   10954 flags.go:33] FLAG: --authentication-skip-lookup="false"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697601   10954 flags.go:33] FLAG: --authentication-token-webhook-cache-ttl="10s"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697607   10954 flags.go:33] FLAG: --authentication-tolerate-lookup-failure="false"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697613   10954 flags.go:33] FLAG: --authorization-always-allow-paths="[/healthz]"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697626   10954 flags.go:33] FLAG: --authorization-kubeconfig=""
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697632   10954 flags.go:33] FLAG: --authorization-webhook-cache-authorized-ttl="10s"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697638   10954 flags.go:33] FLAG: --authorization-webhook-cache-unauthorized-ttl="10s"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697644   10954 flags.go:33] FLAG: --azure-container-registry-config=""
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697649   10954 flags.go:33] FLAG: --bind-address="0.0.0.0"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697655   10954 flags.go:33] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697662   10954 flags.go:33] FLAG: --cert-dir=""
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697667   10954 flags.go:33] FLAG: --cidr-allocator-type="RangeAllocator"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697673   10954 flags.go:33] FLAG: --client-ca-file=""
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697679   10954 flags.go:33] FLAG: --cloud-config=""
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697684   10954 flags.go:33] FLAG: --cloud-provider=""
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697689   10954 flags.go:33] FLAG: --cloud-provider-gce-lb-src-cidrs="130.211.0.0/22,209.85.152.0/22,209.85.204.0/22,35.191.0.0/16"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697700   10954 flags.go:33] FLAG: --cluster-cidr=""
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697705   10954 flags.go:33] FLAG: --cluster-name="kubernetes"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697711   10954 flags.go:33] FLAG: --cluster-signing-cert-file="/etc/kubernetes/ca/ca.pem"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697717   10954 flags.go:33] FLAG: --cluster-signing-key-file="/etc/kubernetes/ca/ca.key"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697723   10954 flags.go:33] FLAG: --concurrent-deployment-syncs="5"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697730   10954 flags.go:33] FLAG: --concurrent-endpoint-syncs="5"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697736   10954 flags.go:33] FLAG: --concurrent-gc-syncs="20"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697742   10954 flags.go:33] FLAG: --concurrent-namespace-syncs="10"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697747   10954 flags.go:33] FLAG: --concurrent-replicaset-syncs="5"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697753   10954 flags.go:33] FLAG: --concurrent-resource-quota-syncs="5"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697762   10954 flags.go:33] FLAG: --concurrent-service-syncs="1"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697778   10954 flags.go:33] FLAG: --concurrent-serviceaccount-token-syncs="5"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697784   10954 flags.go:33] FLAG: --concurrent-ttl-after-finished-syncs="5"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697789   10954 flags.go:33] FLAG: --concurrent_rc_syncs="5"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697795   10954 flags.go:33] FLAG: --configure-cloud-routes="true"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697800   10954 flags.go:33] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697807   10954 flags.go:33] FLAG: --containerd="unix:///var/run/containerd.sock"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697813   10954 flags.go:33] FLAG: --contention-profiling="false"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697819   10954 flags.go:33] FLAG: --controller-start-interval="0s"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697825   10954 flags.go:33] FLAG: --controllers="[*]"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697840   10954 flags.go:33] FLAG: --default-not-ready-toleration-seconds="300"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697846   10954 flags.go:33] FLAG: --default-unreachable-toleration-seconds="300"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697852   10954 flags.go:33] FLAG: --deleting-pods-burst="0"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697858   10954 flags.go:33] FLAG: --deleting-pods-qps="0.1"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697867   10954 flags.go:33] FLAG: --deployment-controller-sync-period="30s"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697873   10954 flags.go:33] FLAG: --disable-attach-detach-reconcile-sync="false"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697879   10954 flags.go:33] FLAG: --docker="unix:///var/run/docker.sock"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697885   10954 flags.go:33] FLAG: --docker-env-metadata-whitelist=""
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697891   10954 flags.go:33] FLAG: --docker-only="false"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697896   10954 flags.go:33] FLAG: --docker-root="/var/lib/docker"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697902   10954 flags.go:33] FLAG: --docker-tls="false"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697908   10954 flags.go:33] FLAG: --docker-tls-ca="ca.pem"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697913   10954 flags.go:33] FLAG: --docker-tls-cert="cert.pem"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697919   10954 flags.go:33] FLAG: --docker-tls-key="key.pem"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697924   10954 flags.go:33] FLAG: --enable-dynamic-provisioning="true"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697930   10954 flags.go:33] FLAG: --enable-garbage-collector="true"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697935   10954 flags.go:33] FLAG: --enable-hostpath-provisioner="false"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697941   10954 flags.go:33] FLAG: --enable-load-reader="false"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697946   10954 flags.go:33] FLAG: --enable-taint-manager="true"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697952   10954 flags.go:33] FLAG: --event-storage-age-limit="default=0"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697958   10954 flags.go:33] FLAG: --event-storage-event-limit="default=0"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697963   10954 flags.go:33] FLAG: --experimental-cluster-signing-duration="8760h0m0s"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697969   10954 flags.go:33] FLAG: --external-cloud-volume-plugin=""
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697981   10954 flags.go:33] FLAG: --feature-gates=""
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697990   10954 flags.go:33] FLAG: --flex-volume-plugin-dir="/usr/libexec/kubernetes/kubelet-plugins/volume/exec/"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.697997   10954 flags.go:33] FLAG: --global-housekeeping-interval="1m0s"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698003   10954 flags.go:33] FLAG: --help="false"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698008   10954 flags.go:33] FLAG: --horizontal-pod-autoscaler-cpu-initialization-period="5m0s"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698014   10954 flags.go:33] FLAG: --horizontal-pod-autoscaler-downscale-delay="5m0s"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698019   10954 flags.go:33] FLAG: --horizontal-pod-autoscaler-downscale-stabilization="5m0s"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698025   10954 flags.go:33] FLAG: --horizontal-pod-autoscaler-initial-readiness-delay="30s"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698030   10954 flags.go:33] FLAG: --horizontal-pod-autoscaler-sync-period="15s"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698036   10954 flags.go:33] FLAG: --horizontal-pod-autoscaler-tolerance="0.1"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698044   10954 flags.go:33] FLAG: --horizontal-pod-autoscaler-upscale-delay="3m0s"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698050   10954 flags.go:33] FLAG: --horizontal-pod-autoscaler-use-rest-clients="true"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698055   10954 flags.go:33] FLAG: --housekeeping-interval="10s"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698061   10954 flags.go:33] FLAG: --http2-max-streams-per-connection="0"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698068   10954 flags.go:33] FLAG: --kube-api-burst="30"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698074   10954 flags.go:33] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698080   10954 flags.go:33] FLAG: --kube-api-qps="20"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698087   10954 flags.go:33] FLAG: --kubeconfig="/etc/kubernetes/controller-manager.kubeconfig"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698093   10954 flags.go:33] FLAG: --large-cluster-size-threshold="50"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698098   10954 flags.go:33] FLAG: --leader-elect="true"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698104   10954 flags.go:33] FLAG: --leader-elect-lease-duration="15s"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698109   10954 flags.go:33] FLAG: --leader-elect-renew-deadline="10s"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698115   10954 flags.go:33] FLAG: --leader-elect-resource-lock="endpoints"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698120   10954 flags.go:33] FLAG: --leader-elect-retry-period="2s"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698126   10954 flags.go:33] FLAG: --log-backtrace-at=":0"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698135   10954 flags.go:33] FLAG: --log-cadvisor-usage="false"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698141   10954 flags.go:33] FLAG: --log-dir=""
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698147   10954 flags.go:33] FLAG: --log-file=""
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698152   10954 flags.go:33] FLAG: --log-flush-frequency="5s"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698158   10954 flags.go:33] FLAG: --logtostderr="true"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698163   10954 flags.go:33] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698170   10954 flags.go:33] FLAG: --master=""
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698181   10954 flags.go:33] FLAG: --mesos-agent="127.0.0.1:5051"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698187   10954 flags.go:33] FLAG: --mesos-agent-timeout="10s"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698193   10954 flags.go:33] FLAG: --min-resync-period="12h0m0s"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698199   10954 flags.go:33] FLAG: --namespace-sync-period="5m0s"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698204   10954 flags.go:33] FLAG: --node-cidr-mask-size="24"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698210   10954 flags.go:33] FLAG: --node-eviction-rate="0.1"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698217   10954 flags.go:33] FLAG: --node-monitor-grace-period="40s"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698222   10954 flags.go:33] FLAG: --node-monitor-period="5s"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698228   10954 flags.go:33] FLAG: --node-startup-grace-period="1m0s"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698234   10954 flags.go:33] FLAG: --node-sync-period="0s"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698240   10954 flags.go:33] FLAG: --pod-eviction-timeout="5m0s"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698246   10954 flags.go:33] FLAG: --port="10252"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698252   10954 flags.go:33] FLAG: --profiling="false"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698257   10954 flags.go:33] FLAG: --pv-recycler-increment-timeout-nfs="30"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698263   10954 flags.go:33] FLAG: --pv-recycler-minimum-timeout-hostpath="60"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698269   10954 flags.go:33] FLAG: --pv-recycler-minimum-timeout-nfs="300"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698274   10954 flags.go:33] FLAG: --pv-recycler-pod-template-filepath-hostpath=""
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698279   10954 flags.go:33] FLAG: --pv-recycler-pod-template-filepath-nfs=""
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698285   10954 flags.go:33] FLAG: --pv-recycler-timeout-increment-hostpath="30"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698290   10954 flags.go:33] FLAG: --pvclaimbinder-sync-period="15s"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698296   10954 flags.go:33] FLAG: --register-retry-count="10"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698301   10954 flags.go:33] FLAG: --requestheader-allowed-names="[]"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698313   10954 flags.go:33] FLAG: --requestheader-client-ca-file=""
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698318   10954 flags.go:33] FLAG: --requestheader-extra-headers-prefix="[x-remote-extra-]"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698330   10954 flags.go:33] FLAG: --requestheader-group-headers="[x-remote-group]"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698340   10954 flags.go:33] FLAG: --requestheader-username-headers="[x-remote-user]"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698350   10954 flags.go:33] FLAG: --resource-quota-sync-period="5m0s"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698356   10954 flags.go:33] FLAG: --root-ca-file="/etc/kubernetes/ssl/ca-kubernetes.crt"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698362   10954 flags.go:33] FLAG: --route-reconciliation-period="10s"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698368   10954 flags.go:33] FLAG: --secondary-node-eviction-rate="0.01"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698375   10954 flags.go:33] FLAG: --secure-port="10257"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698380   10954 flags.go:33] FLAG: --service-account-private-key-file="/etc/kubernetes/ssl/kubernetes-server.key"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698395   10954 flags.go:33] FLAG: --service-cluster-ip-range=""
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698401   10954 flags.go:33] FLAG: --skip-headers="false"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698407   10954 flags.go:33] FLAG: --stderrthreshold="2"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698412   10954 flags.go:33] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698418   10954 flags.go:33] FLAG: --storage-driver-db="cadvisor"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698423   10954 flags.go:33] FLAG: --storage-driver-host="localhost:8086"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698429   10954 flags.go:33] FLAG: --storage-driver-password="root"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698435   10954 flags.go:33] FLAG: --storage-driver-secure="false"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698440   10954 flags.go:33] FLAG: --storage-driver-table="stats"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698446   10954 flags.go:33] FLAG: --storage-driver-user="root"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698451   10954 flags.go:33] FLAG: --terminated-pod-gc-threshold="12500"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698457   10954 flags.go:33] FLAG: --tls-cert-file=""
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698463   10954 flags.go:33] FLAG: --tls-cipher-suites="[]"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698472   10954 flags.go:33] FLAG: --tls-min-version=""
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698477   10954 flags.go:33] FLAG: --tls-private-key-file=""
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698482   10954 flags.go:33] FLAG: --tls-sni-cert-key="[]"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698490   10954 flags.go:33] FLAG: --unhealthy-zone-threshold="0.55"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698497   10954 flags.go:33] FLAG: --use-service-account-credentials="true"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698503   10954 flags.go:33] FLAG: --v="2"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698508   10954 flags.go:33] FLAG: --version="false"
Oct 26 14:00:36 ctl01 kube-controller-manager[10954]: I1026 14:00:36.698517   10954 flags.go:33] FLAG: --vmodule=""
Oct 26 14:00:36 ctl01 systemd[1]: kubelet.service: Dependency Conflicts=cadvisor.service dropped, merged into kubelet.service
Oct 26 14:00:36 ctl01 systemd[1]: kubelet.service: Dependency ConflictedBy=cadvisor.service dropped, merged into kubelet.service
Oct 26 14:00:36 ctl01 systemd[1]: Started ACPI event daemon.
Oct 26 14:00:36 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-enabled', 'kube-controller-manager.service'] in directory '/root'
Oct 26 14:00:36 ctl01 salt-minion[4526]: [INFO    ] {'kube-controller-manager': True}
Oct 26 14:00:36 ctl01 salt-minion[4526]: [INFO    ] Completed state [kube-controller-manager] at time 14:00:36.757780 duration_in_ms=308.237
Oct 26 14:00:36 ctl01 salt-minion[4526]: [INFO    ] Running state [kubectl create ns "netchecker"] at time 14:00:36.758152
Oct 26 14:00:36 ctl01 salt-minion[4526]: [INFO    ] Executing state cmd.run for [kubectl create ns "netchecker"]
Oct 26 14:00:36 ctl01 salt-minion[4526]: [INFO    ] Executing command 'kubectl get ns -o=custom-columns=NAME:.metadata.name | grep -v NAME | grep "netchecker"' in directory '/root'
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.893937   10762 storage_scheduling.go:91] created PriorityClass system-node-critical with value 2000001000
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.898125   10762 storage_scheduling.go:91] created PriorityClass system-cluster-critical with value 2000000000
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.898147   10762 storage_scheduling.go:100] all system priority classes are created successfully or already exist.
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.904629   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/cluster-admin
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.907579   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:discovery
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.918700   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:basic-user
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.922199   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/admin
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.934833   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/edit
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.937971   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/view
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.941487   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:aggregate-to-admin
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.944545   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:aggregate-to-edit
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.948261   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:aggregate-to-view
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.951288   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:heapster
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.961679   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:node
Oct 26 14:00:36 ctl01 kube-proxy[8230]: E1026 14:00:36.964217    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: services is forbidden: User "system:kube-proxy" cannot list resource "services" in API group "" at the cluster scope
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.965243   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:node-problem-detector
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.968679   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:node-proxier
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.972032   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:kubelet-api-admin
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.975189   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:node-bootstrapper
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.978743   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:auth-delegator
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.981933   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:kube-aggregator
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.993257   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:kube-controller-manager
Oct 26 14:00:36 ctl01 kube-apiserver[10762]: I1026 14:00:36.998517   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:kube-scheduler
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.001788   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:kube-dns
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.004816   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:persistent-volume-provisioner
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.007967   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:csi-external-attacher
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.011176   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:aws-cloud-provider
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.014428   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:certificates.k8s.io:certificatesigningrequests:nodeclient
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.017195   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:certificates.k8s.io:certificatesigningrequests:selfnodeclient
Oct 26 14:00:37 ctl01 salt-minion[4526]: [INFO    ] Executing command 'kubectl create ns "netchecker"' in directory '/root'
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.028664   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:volume-scheduler
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.031930   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:csi-external-provisioner
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.035381   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:controller:attachdetach-controller
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.038606   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:controller:clusterrole-aggregation-controller
Oct 26 14:00:37 ctl01 kube-controller-manager[10954]: I1026 14:00:37.039374   10954 serving.go:318] Generated self-signed cert in-memory
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.041739   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:controller:cronjob-controller
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.045673   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:controller:daemon-set-controller
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.049764   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:controller:deployment-controller
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.054489   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:controller:disruption-controller
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.059071   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:controller:endpoint-controller
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.062323   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:controller:expand-controller
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.065356   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:controller:generic-garbage-collector
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.068516   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:controller:horizontal-pod-autoscaler
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.073354   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:controller:job-controller
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.077522   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:controller:namespace-controller
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.081454   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:controller:node-controller
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.086820   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:controller:persistent-volume-binder
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.089967   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:controller:pod-garbage-collector
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.093393   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:controller:replicaset-controller
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.097728   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:controller:replication-controller
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.100755   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:controller:resourcequota-controller
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.104387   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:controller:route-controller
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.111016   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:controller:service-account-controller
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.114252   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:controller:service-controller
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.117784   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:controller:statefulset-controller
Oct 26 14:00:37 ctl01 kube-proxy[8230]: E1026 14:00:37.120890    8230 reflector.go:125] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Endpoints: endpoints is forbidden: User "system:kube-proxy" cannot list resource "endpoints" in API group "" at the cluster scope
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.125390   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:controller:ttl-controller
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.128582   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:controller:certificate-controller
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.131908   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:controller:pvc-protection-controller
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.170580   10762 storage_rbac.go:187] created clusterrole.rbac.authorization.k8s.io/system:controller:pv-protection-controller
Oct 26 14:00:37 ctl01 salt-minion[4526]: [INFO    ] {'pid': 11043, 'retcode': 0, 'stderr': '', 'stdout': 'namespace/netchecker created'}
Oct 26 14:00:37 ctl01 salt-minion[4526]: [INFO    ] Completed state [kubectl create ns "netchecker"] at time 14:00:37.175810 duration_in_ms=417.657
Oct 26 14:00:37 ctl01 salt-minion[4526]: [INFO    ] Running state [/usr/bin/calicoctl] at time 14:00:37.176287
Oct 26 14:00:37 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/usr/bin/calicoctl]
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.211040   10762 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/cluster-admin
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.250463   10762 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:discovery
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.290455   10762 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:basic-user
Oct 26 14:00:37 ctl01 salt-minion[4526]: [INFO    ] File /usr/bin/calicoctl is in the correct state
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.330727   10762 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:node-proxier
Oct 26 14:00:37 ctl01 salt-minion[4526]: [INFO    ] Completed state [/usr/bin/calicoctl] at time 14:00:37.330885 duration_in_ms=154.597
Oct 26 14:00:37 ctl01 salt-minion[4526]: [INFO    ] Running state [/usr/bin/birdcl] at time 14:00:37.331173
Oct 26 14:00:37 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/usr/bin/birdcl]
Oct 26 14:00:37 ctl01 salt-minion[4526]: [INFO    ] File /usr/bin/birdcl is in the correct state
Oct 26 14:00:37 ctl01 salt-minion[4526]: [INFO    ] Completed state [/usr/bin/birdcl] at time 14:00:37.334979 duration_in_ms=3.806
Oct 26 14:00:37 ctl01 salt-minion[4526]: [INFO    ] Running state [/opt/cni/bin/calico] at time 14:00:37.335212
Oct 26 14:00:37 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/opt/cni/bin/calico]
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.370607   10762 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:kube-controller-manager
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.410927   10762 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:kube-dns
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.450523   10762 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:kube-scheduler
Oct 26 14:00:37 ctl01 salt-minion[4526]: [INFO    ] File /opt/cni/bin/calico is in the correct state
Oct 26 14:00:37 ctl01 salt-minion[4526]: [INFO    ] Completed state [/opt/cni/bin/calico] at time 14:00:37.455490 duration_in_ms=120.277
Oct 26 14:00:37 ctl01 salt-minion[4526]: [INFO    ] Running state [/opt/cni/bin/calico-ipam] at time 14:00:37.455791
Oct 26 14:00:37 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/opt/cni/bin/calico-ipam]
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.490400   10762 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:aws-cloud-provider
Oct 26 14:00:37 ctl01 kube-scheduler[10878]: I1026 14:00:37.509429   10878 serving.go:318] Generated self-signed cert in-memory
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.531400   10762 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:node
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.570398   10762 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:volume-scheduler
Oct 26 14:00:37 ctl01 salt-minion[4526]: [INFO    ] File /opt/cni/bin/calico-ipam is in the correct state
Oct 26 14:00:37 ctl01 salt-minion[4526]: [INFO    ] Completed state [/opt/cni/bin/calico-ipam] at time 14:00:37.572195 duration_in_ms=116.403
Oct 26 14:00:37 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/cni/net.d/10-calico.conf] at time 14:00:37.572489
Oct 26 14:00:37 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/cni/net.d/10-calico.conf]
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.610546   10762 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:attachdetach-controller
Oct 26 14:00:37 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.650750   10762 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:clusterrole-aggregation-controller
Oct 26 14:00:37 ctl01 kube-controller-manager[10954]: W1026 14:00:37.660278   10954 authentication.go:249] No authentication-kubeconfig provided in order to lookup client-ca-file in configmap/extension-apiserver-authentication in kube-system, so client certificate authentication won't work.
Oct 26 14:00:37 ctl01 kube-controller-manager[10954]: W1026 14:00:37.660374   10954 authentication.go:252] No authentication-kubeconfig provided in order to lookup requestheader-client-ca-file in configmap/extension-apiserver-authentication in kube-system, so request-header client certificate authentication won't work.
Oct 26 14:00:37 ctl01 kube-controller-manager[10954]: W1026 14:00:37.660401   10954 authorization.go:146] No authorization-kubeconfig provided, so SubjectAccessReview of authorization tokens won't work.
Oct 26 14:00:37 ctl01 kube-controller-manager[10954]: I1026 14:00:37.660427   10954 controllermanager.go:151] Version: v1.13.5-3+98374c02d2d8c1
Oct 26 14:00:37 ctl01 kube-controller-manager[10954]: I1026 14:00:37.660902   10954 secure_serving.go:116] Serving securely on [::]:10257
Oct 26 14:00:37 ctl01 kube-controller-manager[10954]: I1026 14:00:37.661356   10954 deprecated_insecure_serving.go:51] Serving insecurely on [::]:10252
Oct 26 14:00:37 ctl01 kube-controller-manager[10954]: I1026 14:00:37.661557   10954 leaderelection.go:205] attempting to acquire leader lease  kube-system/kube-controller-manager...
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.691442   10762 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:cronjob-controller
Oct 26 14:00:37 ctl01 kube-controller-manager[10954]: I1026 14:00:37.707154   10954 leaderelection.go:214] successfully acquired lease kube-system/kube-controller-manager
Oct 26 14:00:37 ctl01 kube-controller-manager[10954]: I1026 14:00:37.707202   10954 event.go:221] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"kube-controller-manager", UID:"fcd193af-f7f8-11e9-8756-525400d144f2", APIVersion:"v1", ResourceVersion:"159", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' ctl01_fccae1df-f7f8-11e9-8906-525400d144f2 became leader
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.730736   10762 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:daemon-set-controller
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.770442   10762 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:deployment-controller
Oct 26 14:00:37 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.810703   10762 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:disruption-controller
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.850906   10762 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:endpoint-controller
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.890881   10762 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:expand-controller
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.930693   10762 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:generic-garbage-collector
Oct 26 14:00:37 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:37 ctl01 kube-apiserver[10762]: I1026 14:00:37.973185   10762 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:horizontal-pod-autoscaler
Oct 26 14:00:38 ctl01 kube-apiserver[10762]: I1026 14:00:38.010683   10762 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:job-controller
Oct 26 14:00:38 ctl01 kube-proxy[8230]: I1026 14:00:38.020407    8230 controller_utils.go:1034] Caches are synced for service config controller
Oct 26 14:00:38 ctl01 kube-proxy[8230]: I1026 14:00:38.020504    8230 proxier.go:645] Not syncing iptables until Services and Endpoints have been received from master
Oct 26 14:00:38 ctl01 kube-apiserver[10762]: I1026 14:00:38.050586   10762 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:namespace-controller
Oct 26 14:00:38 ctl01 kube-apiserver[10762]: I1026 14:00:38.090711   10762 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:node-controller
Oct 26 14:00:38 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:38 ctl01 kube-apiserver[10762]: I1026 14:00:38.132547   10762 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:persistent-volume-binder
Oct 26 14:00:38 ctl01 kube-apiserver[10762]: I1026 14:00:38.170515   10762 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:pod-garbage-collector
Oct 26 14:00:38 ctl01 kube-apiserver[10762]: I1026 14:00:38.211188   10762 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:replicaset-controller
Oct 26 14:00:38 ctl01 kube-proxy[8230]: I1026 14:00:38.220314    8230 controller_utils.go:1034] Caches are synced for endpoints config controller
Oct 26 14:00:38 ctl01 kube-proxy[8230]: I1026 14:00:38.220432    8230 service.go:309] Adding new service port "default/kubernetes:https" at 10.254.0.1:443/TCP
Oct 26 14:00:38 ctl01 kube-apiserver[10762]: I1026 14:00:38.251537   10762 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:replication-controller
Oct 26 14:00:38 ctl01 salt-minion[4526]: [INFO    ] File /etc/cni/net.d/10-calico.conf is in the correct state
Oct 26 14:00:38 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/cni/net.d/10-calico.conf] at time 14:00:38.259246 duration_in_ms=686.755
Oct 26 14:00:38 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/calico/network-environment] at time 14:00:38.259597
Oct 26 14:00:38 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/calico/network-environment]
Oct 26 14:00:38 ctl01 kube-apiserver[10762]: I1026 14:00:38.290715   10762 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:resourcequota-controller
Oct 26 14:00:38 ctl01 kube-scheduler[10878]: W1026 14:00:38.303529   10878 authentication.go:249] No authentication-kubeconfig provided in order to lookup client-ca-file in configmap/extension-apiserver-authentication in kube-system, so client certificate authentication won't work.
Oct 26 14:00:38 ctl01 kube-scheduler[10878]: W1026 14:00:38.303618   10878 authentication.go:252] No authentication-kubeconfig provided in order to lookup requestheader-client-ca-file in configmap/extension-apiserver-authentication in kube-system, so request-header client certificate authentication won't work.
Oct 26 14:00:38 ctl01 kube-scheduler[10878]: W1026 14:00:38.303654   10878 authorization.go:146] No authorization-kubeconfig provided, so SubjectAccessReview of authorization tokens won't work.
Oct 26 14:00:38 ctl01 kube-scheduler[10878]: I1026 14:00:38.317514   10878 server.go:150] Version: v1.13.5-3+98374c02d2d8c1
Oct 26 14:00:38 ctl01 kube-scheduler[10878]: I1026 14:00:38.317570   10878 defaults.go:210] TaintNodesByCondition is enabled, PodToleratesNodeTaints predicate is mandatory
Oct 26 14:00:38 ctl01 kube-scheduler[10878]: I1026 14:00:38.317983   10878 factory.go:1157] Creating scheduler from algorithm provider 'DefaultProvider'
Oct 26 14:00:38 ctl01 kube-scheduler[10878]: I1026 14:00:38.318024   10878 factory.go:1256] Creating scheduler with fit predicates 'map[MaxEBSVolumeCount:{} MaxGCEPDVolumeCount:{} MaxAzureDiskVolumeCount:{} NoVolumeZoneConflict:{} PodToleratesNodeTaints:{} NoDiskConflict:{} CheckVolumeBinding:{} MaxCSIVolumeCountPred:{} MatchInterPodAffinity:{} GeneralPredicates:{} CheckNodeUnschedulable:{}]' and priority functions 'map[SelectorSpreadPriority:{} InterPodAffinityPriority:{} LeastRequestedPriority:{} BalancedResourceAllocation:{} NodePreferAvoidPodsPriority:{} NodeAffinityPriority:{} TaintTolerationPriority:{} ImageLocalityPriority:{}]'
Oct 26 14:00:38 ctl01 kube-scheduler[10878]: W1026 14:00:38.318843   10878 authorization.go:47] Authorization is disabled
Oct 26 14:00:38 ctl01 kube-scheduler[10878]: W1026 14:00:38.318863   10878 authentication.go:55] Authentication is disabled
Oct 26 14:00:38 ctl01 kube-scheduler[10878]: I1026 14:00:38.318874   10878 deprecated_insecure_serving.go:49] Serving healthz insecurely on [::]:10251
Oct 26 14:00:38 ctl01 kube-scheduler[10878]: I1026 14:00:38.319481   10878 secure_serving.go:116] Serving securely on [::]:10259
Oct 26 14:00:38 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:38 ctl01 kube-apiserver[10762]: I1026 14:00:38.359619   10762 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:route-controller
Oct 26 14:00:38 ctl01 kube-apiserver[10762]: I1026 14:00:38.378840   10762 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:service-account-controller
Oct 26 14:00:38 ctl01 kube-apiserver[10762]: I1026 14:00:38.410907   10762 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:service-controller
Oct 26 14:00:38 ctl01 kube-apiserver[10762]: I1026 14:00:38.451033   10762 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:statefulset-controller
Oct 26 14:00:38 ctl01 kube-apiserver[10762]: I1026 14:00:38.494564   10762 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:ttl-controller
Oct 26 14:00:38 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:38 ctl01 kube-apiserver[10762]: I1026 14:00:38.530850   10762 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:certificate-controller
Oct 26 14:00:38 ctl01 kube-apiserver[10762]: I1026 14:00:38.571006   10762 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:pvc-protection-controller
Oct 26 14:00:38 ctl01 kube-apiserver[10762]: I1026 14:00:38.612263   10762 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:pv-protection-controller
Oct 26 14:00:38 ctl01 kube-apiserver[10762]: I1026 14:00:38.650164   10762 controller.go:608] quota admission added evaluator for: roles.rbac.authorization.k8s.io
Oct 26 14:00:38 ctl01 kube-apiserver[10762]: I1026 14:00:38.653856   10762 storage_rbac.go:246] created role.rbac.authorization.k8s.io/extension-apiserver-authentication-reader in kube-system
Oct 26 14:00:38 ctl01 salt-minion[4526]: [INFO    ] File /etc/calico/network-environment is in the correct state
Oct 26 14:00:38 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/calico/network-environment] at time 14:00:38.665280 duration_in_ms=405.682
Oct 26 14:00:38 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/calico/calicoctl.cfg] at time 14:00:38.665685
Oct 26 14:00:38 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/calico/calicoctl.cfg]
Oct 26 14:00:38 ctl01 kube-apiserver[10762]: I1026 14:00:38.690668   10762 storage_rbac.go:246] created role.rbac.authorization.k8s.io/system:controller:bootstrap-signer in kube-system
Oct 26 14:00:38 ctl01 kube-apiserver[10762]: I1026 14:00:38.731548   10762 storage_rbac.go:246] created role.rbac.authorization.k8s.io/system:controller:cloud-provider in kube-system
Oct 26 14:00:38 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:38 ctl01 kube-apiserver[10762]: I1026 14:00:38.771140   10762 storage_rbac.go:246] created role.rbac.authorization.k8s.io/system:controller:token-cleaner in kube-system
Oct 26 14:00:38 ctl01 kube-apiserver[10762]: I1026 14:00:38.811014   10762 storage_rbac.go:246] created role.rbac.authorization.k8s.io/system::leader-locking-kube-controller-manager in kube-system
Oct 26 14:00:38 ctl01 kube-apiserver[10762]: I1026 14:00:38.851144   10762 storage_rbac.go:246] created role.rbac.authorization.k8s.io/system::leader-locking-kube-scheduler in kube-system
Oct 26 14:00:38 ctl01 kube-apiserver[10762]: I1026 14:00:38.892951   10762 storage_rbac.go:246] created role.rbac.authorization.k8s.io/system:controller:bootstrap-signer in kube-public
Oct 26 14:00:38 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:38 ctl01 kube-apiserver[10762]: I1026 14:00:38.929072   10762 controller.go:608] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io
Oct 26 14:00:38 ctl01 kube-apiserver[10762]: I1026 14:00:38.930764   10762 storage_rbac.go:276] created rolebinding.rbac.authorization.k8s.io/system::leader-locking-kube-controller-manager in kube-system
Oct 26 14:00:38 ctl01 kube-apiserver[10762]: I1026 14:00:38.971442   10762 storage_rbac.go:276] created rolebinding.rbac.authorization.k8s.io/system::leader-locking-kube-scheduler in kube-system
Oct 26 14:00:39 ctl01 kube-apiserver[10762]: I1026 14:00:39.012714   10762 storage_rbac.go:276] created rolebinding.rbac.authorization.k8s.io/system:controller:bootstrap-signer in kube-system
Oct 26 14:00:39 ctl01 kube-apiserver[10762]: I1026 14:00:39.051726   10762 storage_rbac.go:276] created rolebinding.rbac.authorization.k8s.io/system:controller:cloud-provider in kube-system
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] File /etc/calico/calicoctl.cfg is in the correct state
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/calico/calicoctl.cfg] at time 14:00:39.061184 duration_in_ms=395.498
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/systemd/system/calico-node.service] at time 14:00:39.062670
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/systemd/system/calico-node.service]
Oct 26 14:00:39 ctl01 kube-apiserver[10762]: I1026 14:00:39.091709   10762 storage_rbac.go:276] created rolebinding.rbac.authorization.k8s.io/system:controller:token-cleaner in kube-system
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] File /etc/systemd/system/calico-node.service is in the correct state
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/systemd/system/calico-node.service] at time 14:00:39.103871 duration_in_ms=41.201
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] Running state [/var/lib/calico] at time 14:00:39.104193
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] Executing state file.directory for [/var/lib/calico]
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] Directory /var/lib/calico is in the correct state
Oct 26 14:00:39 ctl01 salt-minion[4526]: Directory /var/lib/calico updated
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] Completed state [/var/lib/calico] at time 14:00:39.105743 duration_in_ms=1.55
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] Running state [/var/log/calico] at time 14:00:39.106028
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] Executing state file.directory for [/var/log/calico]
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] Directory /var/log/calico is in the correct state
Oct 26 14:00:39 ctl01 salt-minion[4526]: Directory /var/log/calico updated
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] Completed state [/var/log/calico] at time 14:00:39.107532 duration_in_ms=1.504
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] Running state [calico-node] at time 14:00:39.113231
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] Executing state service.running for [calico-node]
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'status', 'calico-node.service', '-n', '0'] in directory '/root'
Oct 26 14:00:39 ctl01 kube-apiserver[10762]: I1026 14:00:39.130710   10762 storage_rbac.go:276] created rolebinding.rbac.authorization.k8s.io/system:controller:bootstrap-signer in kube-public
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-active', 'calico-node.service'] in directory '/root'
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-enabled', 'calico-node.service'] in directory '/root'
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] The service calico-node is already running
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] Completed state [calico-node] at time 14:00:39.175720 duration_in_ms=62.488
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/proxy.kubeconfig] at time 14:00:39.176492
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/kubernetes/proxy.kubeconfig]
Oct 26 14:00:39 ctl01 kube-scheduler[10878]: I1026 14:00:39.233685   10878 controller_utils.go:1027] Waiting for caches to sync for scheduler controller
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:39 ctl01 kube-scheduler[10878]: I1026 14:00:39.334379   10878 controller_utils.go:1034] Caches are synced for scheduler controller
Oct 26 14:00:39 ctl01 kube-scheduler[10878]: I1026 14:00:39.334507   10878 leaderelection.go:205] attempting to acquire leader lease  kube-system/kube-scheduler...
Oct 26 14:00:39 ctl01 kube-scheduler[10878]: I1026 14:00:39.346346   10878 leaderelection.go:214] successfully acquired lease kube-system/kube-scheduler
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:39 ctl01 kube-controller-manager[10954]: I1026 14:00:39.734413   10954 plugins.go:103] No cloud provider specified.
Oct 26 14:00:39 ctl01 kube-controller-manager[10954]: I1026 14:00:39.738846   10954 controllermanager.go:501] Starting "csrcleaner"
Oct 26 14:00:39 ctl01 kube-controller-manager[10954]: I1026 14:00:39.738921   10954 controller_utils.go:1027] Waiting for caches to sync for tokens controller
Oct 26 14:00:39 ctl01 kube-apiserver[10762]: I1026 14:00:39.748102   10762 controller.go:608] quota admission added evaluator for: serviceaccounts
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:39 ctl01 kube-controller-manager[10954]: I1026 14:00:39.839406   10954 controller_utils.go:1034] Caches are synced for tokens controller
Oct 26 14:00:39 ctl01 kube-controller-manager[10954]: I1026 14:00:39.882307   10954 controllermanager.go:516] Started "csrcleaner"
Oct 26 14:00:39 ctl01 kube-controller-manager[10954]: I1026 14:00:39.882326   10954 controllermanager.go:501] Starting "nodeipam"
Oct 26 14:00:39 ctl01 kube-controller-manager[10954]: W1026 14:00:39.882334   10954 controllermanager.go:508] Skipping "nodeipam"
Oct 26 14:00:39 ctl01 kube-controller-manager[10954]: I1026 14:00:39.882341   10954 controllermanager.go:501] Starting "service"
Oct 26 14:00:39 ctl01 kube-controller-manager[10954]: I1026 14:00:39.882448   10954 cleaner.go:81] Starting CSR cleaner controller
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] File /etc/kubernetes/proxy.kubeconfig is in the correct state
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/proxy.kubeconfig] at time 14:00:39.897756 duration_in_ms=721.264
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/systemd/system/kube-proxy.service] at time 14:00:39.910885
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/systemd/system/kube-proxy.service]
Oct 26 14:00:39 ctl01 kube-controller-manager[10954]: E1026 14:00:39.939762   10954 core.go:76] Failed to start service controller: WARNING: no cloud provider provided, services of type LoadBalancer will fail
Oct 26 14:00:39 ctl01 kube-controller-manager[10954]: W1026 14:00:39.940172   10954 controllermanager.go:508] Skipping "service"
Oct 26 14:00:39 ctl01 kube-controller-manager[10954]: I1026 14:00:39.940508   10954 controllermanager.go:501] Starting "namespace"
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] File /etc/systemd/system/kube-proxy.service is in the correct state
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/systemd/system/kube-proxy.service] at time 14:00:39.944018 duration_in_ms=33.133
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/default/kube-proxy] at time 14:00:39.944289
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/default/kube-proxy]
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] File /etc/default/kube-proxy is in the correct state
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/default/kube-proxy] at time 14:00:39.946114 duration_in_ms=1.825
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] Running state [kube-proxy] at time 14:00:39.948639
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] Executing state service.running for [kube-proxy]
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'status', 'kube-proxy.service', '-n', '0'] in directory '/root'
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-active', 'kube-proxy.service'] in directory '/root'
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'is-enabled', 'kube-proxy.service'] in directory '/root'
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] The service kube-proxy is already running
Oct 26 14:00:39 ctl01 salt-minion[4526]: [INFO    ] Completed state [kube-proxy] at time 14:00:39.998537 duration_in_ms=49.898
Oct 26 14:00:40 ctl01 salt-minion[4526]: [INFO    ] Running state [/srv/kubernetes] at time 14:00:40.000438
Oct 26 14:00:40 ctl01 salt-minion[4526]: [INFO    ] Executing state file.directory for [/srv/kubernetes]
Oct 26 14:00:40 ctl01 salt-minion[4526]: [INFO    ] Directory /srv/kubernetes is in the correct state
Oct 26 14:00:40 ctl01 salt-minion[4526]: Directory /srv/kubernetes updated
Oct 26 14:00:40 ctl01 salt-minion[4526]: [INFO    ] Completed state [/srv/kubernetes] at time 14:00:40.001927 duration_in_ms=1.489
Oct 26 14:00:40 ctl01 salt-minion[4526]: [INFO    ] Running state [/srv/kubernetes/roles/cluster-admin/root-cluster-admin-binding-rolebinding.yml] at time 14:00:40.002761
Oct 26 14:00:40 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/srv/kubernetes/roles/cluster-admin/root-cluster-admin-binding-rolebinding.yml]
Oct 26 14:00:40 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/rolebinding.yml'
Oct 26 14:00:40 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 14:00:40 ctl01 salt-minion[4526]: New file
Oct 26 14:00:40 ctl01 salt-minion[4526]: [INFO    ] Completed state [/srv/kubernetes/roles/cluster-admin/root-cluster-admin-binding-rolebinding.yml] at time 14:00:40.037284 duration_in_ms=34.523
Oct 26 14:00:40 ctl01 salt-minion[4526]: [INFO    ] Running state [kubectl apply -f /srv/kubernetes/roles/cluster-admin/root-cluster-admin-binding-rolebinding.yml] at time 14:00:40.038640
Oct 26 14:00:40 ctl01 salt-minion[4526]: [INFO    ] Executing state cmd.run for [kubectl apply -f /srv/kubernetes/roles/cluster-admin/root-cluster-admin-binding-rolebinding.yml]
Oct 26 14:00:40 ctl01 salt-minion[4526]: [INFO    ] Executing command 'kubectl apply -f /srv/kubernetes/roles/cluster-admin/root-cluster-admin-binding-rolebinding.yml' in directory '/root'
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.041709   10954 controllermanager.go:516] Started "namespace"
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.041736   10954 controllermanager.go:501] Starting "serviceaccount"
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.041858   10954 namespace_controller.go:186] Starting namespace controller
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.041871   10954 controller_utils.go:1027] Waiting for caches to sync for namespace controller
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.110237   10954 controllermanager.go:516] Started "serviceaccount"
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.110285   10954 serviceaccounts_controller.go:115] Starting service account controller
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.110326   10954 controller_utils.go:1027] Waiting for caches to sync for service account controller
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.110287   10954 controllermanager.go:501] Starting "daemonset"
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.159824   10954 controllermanager.go:516] Started "daemonset"
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.159861   10954 controllermanager.go:501] Starting "cronjob"
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.159886   10954 daemon_controller.go:269] Starting daemon sets controller
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.159905   10954 controller_utils.go:1027] Waiting for caches to sync for daemon sets controller
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.215779   10954 controllermanager.go:516] Started "cronjob"
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.215811   10954 cronjob_controller.go:92] Starting CronJob Manager
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: W1026 14:00:40.215820   10954 controllermanager.go:495] "tokencleaner" is disabled
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.215869   10954 controllermanager.go:501] Starting "persistentvolume-expander"
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.257709   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/aws-ebs"
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.257750   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/gce-pd"
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.257763   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/cinder"
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.257782   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/portworx-volume"
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.257793   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/vsphere-volume"
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.257803   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/glusterfs"
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.257832   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/rbd"
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.257844   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/azure-disk"
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.257856   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/azure-file"
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.257866   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/photon-pd"
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.257880   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/scaleio"
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.257903   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/storageos"
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.257914   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/fc"
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.258162   10954 controllermanager.go:516] Started "persistentvolume-expander"
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.258177   10954 controllermanager.go:501] Starting "horizontalpodautoscaling"
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.258311   10954 expand_controller.go:153] Starting expand controller
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.258323   10954 controller_utils.go:1027] Waiting for caches to sync for expand controller
Oct 26 14:00:40 ctl01 salt-minion[4526]: [INFO    ] {'pid': 11350, 'retcode': 0, 'stderr': '', 'stdout': 'clusterrolebinding.rbac.authorization.k8s.io/root-cluster-admin-binding created'}
Oct 26 14:00:40 ctl01 salt-minion[4526]: [INFO    ] Completed state [kubectl apply -f /srv/kubernetes/roles/cluster-admin/root-cluster-admin-binding-rolebinding.yml] at time 14:00:40.387558 duration_in_ms=348.914
Oct 26 14:00:40 ctl01 salt-minion[4526]: [INFO    ] Returning information for job: 20191026135955807384
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.692306   10954 controllermanager.go:516] Started "horizontalpodautoscaling"
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.692377   10954 controllermanager.go:501] Starting "ttl"
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.692346   10954 horizontal.go:156] Starting HPA controller
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.692465   10954 controller_utils.go:1027] Waiting for caches to sync for HPA controller
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.945838   10954 controllermanager.go:516] Started "ttl"
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.945860   10954 ttl_controller.go:116] Starting TTL controller
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.945884   10954 controller_utils.go:1027] Waiting for caches to sync for TTL controller
Oct 26 14:00:40 ctl01 kube-controller-manager[10954]: I1026 14:00:40.945874   10954 controllermanager.go:501] Starting "persistentvolume-binder"
Oct 26 14:00:41 ctl01 salt-minion[4526]: [INFO    ] User sudo_ubuntu Executing command state.sls with jid 20191026140041088885
Oct 26 14:00:41 ctl01 salt-minion[4526]: [INFO    ] Starting a new job with PID 11372
Oct 26 14:00:41 ctl01 kube-controller-manager[10954]: I1026 14:00:41.200565   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/host-path"
Oct 26 14:00:41 ctl01 kube-controller-manager[10954]: I1026 14:00:41.200616   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/nfs"
Oct 26 14:00:41 ctl01 kube-controller-manager[10954]: I1026 14:00:41.200632   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/glusterfs"
Oct 26 14:00:41 ctl01 kube-controller-manager[10954]: I1026 14:00:41.200645   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/rbd"
Oct 26 14:00:41 ctl01 kube-controller-manager[10954]: I1026 14:00:41.200657   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/quobyte"
Oct 26 14:00:41 ctl01 kube-controller-manager[10954]: I1026 14:00:41.200670   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/azure-file"
Oct 26 14:00:41 ctl01 kube-controller-manager[10954]: I1026 14:00:41.200683   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/flocker"
Oct 26 14:00:41 ctl01 kube-controller-manager[10954]: I1026 14:00:41.200704   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/portworx-volume"
Oct 26 14:00:41 ctl01 kube-controller-manager[10954]: I1026 14:00:41.200723   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/scaleio"
Oct 26 14:00:41 ctl01 kube-controller-manager[10954]: I1026 14:00:41.200737   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/local-volume"
Oct 26 14:00:41 ctl01 kube-controller-manager[10954]: I1026 14:00:41.200750   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/storageos"
Oct 26 14:00:41 ctl01 kube-controller-manager[10954]: I1026 14:00:41.200762   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/aws-ebs"
Oct 26 14:00:41 ctl01 kube-controller-manager[10954]: I1026 14:00:41.200775   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/gce-pd"
Oct 26 14:00:41 ctl01 kube-controller-manager[10954]: I1026 14:00:41.200792   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/cinder"
Oct 26 14:00:41 ctl01 kube-controller-manager[10954]: I1026 14:00:41.200805   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/vsphere-volume"
Oct 26 14:00:41 ctl01 kube-controller-manager[10954]: I1026 14:00:41.200818   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/azure-disk"
Oct 26 14:00:41 ctl01 kube-controller-manager[10954]: I1026 14:00:41.200830   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/photon-pd"
Oct 26 14:00:41 ctl01 kube-controller-manager[10954]: I1026 14:00:41.200892   10954 controllermanager.go:516] Started "persistentvolume-binder"
Oct 26 14:00:41 ctl01 kube-controller-manager[10954]: I1026 14:00:41.200909   10954 controllermanager.go:501] Starting "root-ca-cert-publisher"
Oct 26 14:00:41 ctl01 kube-controller-manager[10954]: W1026 14:00:41.200920   10954 controllermanager.go:508] Skipping "root-ca-cert-publisher"
Oct 26 14:00:41 ctl01 kube-controller-manager[10954]: I1026 14:00:41.200929   10954 controllermanager.go:501] Starting "podgc"
Oct 26 14:00:41 ctl01 kube-controller-manager[10954]: I1026 14:00:41.200975   10954 pv_controller_base.go:271] Starting persistent volume controller
Oct 26 14:00:41 ctl01 kube-controller-manager[10954]: I1026 14:00:41.201010   10954 controller_utils.go:1027] Waiting for caches to sync for persistent volume controller
Oct 26 14:00:41 ctl01 kube-controller-manager[10954]: I1026 14:00:41.448666   10954 controllermanager.go:516] Started "podgc"
Oct 26 14:00:41 ctl01 kube-controller-manager[10954]: I1026 14:00:41.448699   10954 gc_controller.go:76] Starting GC controller
Oct 26 14:00:41 ctl01 kube-controller-manager[10954]: I1026 14:00:41.448719   10954 controller_utils.go:1027] Waiting for caches to sync for GC controller
Oct 26 14:00:41 ctl01 kube-controller-manager[10954]: I1026 14:00:41.448700   10954 controllermanager.go:501] Starting "garbagecollector"
Oct 26 14:00:41 ctl01 kube-controller-manager[10954]: W1026 14:00:41.841951   10954 garbagecollector.go:649] failed to discover preferred resources: the cache has not been filled yet
Oct 26 14:00:41 ctl01 kube-controller-manager[10954]: I1026 14:00:41.842536   10954 garbagecollector.go:133] Starting garbage collector controller
Oct 26 14:00:41 ctl01 kube-controller-manager[10954]: I1026 14:00:41.842564   10954 controller_utils.go:1027] Waiting for caches to sync for garbage collector controller
Oct 26 14:00:41 ctl01 kube-controller-manager[10954]: I1026 14:00:41.842585   10954 controllermanager.go:516] Started "garbagecollector"
Oct 26 14:00:41 ctl01 kube-controller-manager[10954]: I1026 14:00:41.842625   10954 controllermanager.go:501] Starting "deployment"
Oct 26 14:00:41 ctl01 kube-controller-manager[10954]: I1026 14:00:41.842591   10954 graph_builder.go:308] GraphBuilder running
Oct 26 14:00:41 ctl01 kube-controller-manager[10954]: I1026 14:00:41.855578   10954 garbagecollector.go:204] syncing garbage collector with updated resources from discovery (attempt 1): added: [/v1, Resource=configmaps /v1, Resource=endpoints /v1, Resource=events /v1, Resource=limitranges /v1, Resource=namespaces /v1, Resource=nodes /v1, Resource=persistentvolumeclaims /v1, Resource=persistentvolumes /v1, Resource=pods /v1, Resource=podtemplates /v1, Resource=replicationcontrollers /v1, Resource=resourcequotas /v1, Resource=secrets /v1, Resource=serviceaccounts /v1, Resource=services admissionregistration.k8s.io/v1beta1, Resource=mutatingwebhookconfigurations admissionregistration.k8s.io/v1beta1, Resource=validatingwebhookconfigurations apiextensions.k8s.io/v1beta1, Resource=customresourcedefinitions apiregistration.k8s.io/v1, Resource=apiservices apps/v1, Resource=controllerrevisions apps/v1, Resource=daemonsets apps/v1, Resource=deployments apps/v1, Resource=replicasets apps/v1, Resource=statefulsets autoscaling/v1, Resource=horizontalpodautoscalers batch/v1, Resource=jobs batch/v1beta1, Resource=cronjobs certificates.k8s.io/v1beta1, Resource=certificatesigningrequests coordination.k8s.io/v1beta1, Resource=leases events.k8s.io/v1beta1, Resource=events extensions/v1beta1, Resource=daemonsets extensions/v1beta1, Resource=deployments extensions/v1beta1, Resource=ingresses extensions/v1beta1, Resource=networkpolicies extensions/v1beta1, Resource=podsecuritypolicies extensions/v1beta1, Resource=replicasets networking.k8s.io/v1, Resource=networkpolicies policy/v1beta1, Resource=poddisruptionbudgets policy/v1beta1, Resource=podsecuritypolicies rbac.authorization.k8s.io/v1, Resource=clusterrolebindings rbac.authorization.k8s.io/v1, Resource=clusterroles rbac.authorization.k8s.io/v1, Resource=rolebindings rbac.authorization.k8s.io/v1, Resource=roles scheduling.k8s.io/v1beta1, Resource=priorityclasses storage.k8s.io/v1, Resource=storageclasses storage.k8s.io/v1, Resource=volumeattachments], removed: []
Oct 26 14:00:42 ctl01 kube-controller-manager[10954]: I1026 14:00:42.096842   10954 controllermanager.go:516] Started "deployment"
Oct 26 14:00:42 ctl01 kube-controller-manager[10954]: I1026 14:00:42.096885   10954 controllermanager.go:501] Starting "clusterrole-aggregation"
Oct 26 14:00:42 ctl01 kube-controller-manager[10954]: I1026 14:00:42.096930   10954 deployment_controller.go:152] Starting deployment controller
Oct 26 14:00:42 ctl01 kube-controller-manager[10954]: I1026 14:00:42.096942   10954 controller_utils.go:1027] Waiting for caches to sync for deployment controller
Oct 26 14:00:42 ctl01 kube-controller-manager[10954]: I1026 14:00:42.352740   10954 controllermanager.go:516] Started "clusterrole-aggregation"
Oct 26 14:00:42 ctl01 kube-controller-manager[10954]: I1026 14:00:42.352804   10954 clusterroleaggregation_controller.go:148] Starting ClusterRoleAggregator
Oct 26 14:00:42 ctl01 kube-controller-manager[10954]: I1026 14:00:42.352819   10954 controller_utils.go:1027] Waiting for caches to sync for ClusterRoleAggregator controller
Oct 26 14:00:42 ctl01 kube-controller-manager[10954]: I1026 14:00:42.352778   10954 controllermanager.go:501] Starting "pv-protection"
Oct 26 14:00:42 ctl01 kube-controller-manager[10954]: I1026 14:00:42.596065   10954 controllermanager.go:516] Started "pv-protection"
Oct 26 14:00:42 ctl01 kube-controller-manager[10954]: I1026 14:00:42.596100   10954 controllermanager.go:501] Starting "replicaset"
Oct 26 14:00:42 ctl01 kube-controller-manager[10954]: I1026 14:00:42.596103   10954 pv_protection_controller.go:81] Starting PV protection controller
Oct 26 14:00:42 ctl01 kube-controller-manager[10954]: I1026 14:00:42.596119   10954 controller_utils.go:1027] Waiting for caches to sync for PV protection controller
Oct 26 14:00:42 ctl01 kube-controller-manager[10954]: I1026 14:00:42.846719   10954 controllermanager.go:516] Started "replicaset"
Oct 26 14:00:42 ctl01 kube-controller-manager[10954]: I1026 14:00:42.846747   10954 replica_set.go:182] Starting replicaset controller
Oct 26 14:00:42 ctl01 kube-controller-manager[10954]: I1026 14:00:42.846764   10954 controller_utils.go:1027] Waiting for caches to sync for ReplicaSet controller
Oct 26 14:00:42 ctl01 kube-controller-manager[10954]: I1026 14:00:42.846750   10954 controllermanager.go:501] Starting "disruption"
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: I1026 14:00:43.101057   10954 controllermanager.go:516] Started "disruption"
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: W1026 14:00:43.101104   10954 controllermanager.go:495] "bootstrapsigner" is disabled
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: I1026 14:00:43.101119   10954 controllermanager.go:501] Starting "csrsigning"
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: W1026 14:00:43.101163   10954 controllermanager.go:508] Skipping "csrsigning"
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: I1026 14:00:43.101175   10954 controllermanager.go:501] Starting "route"
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: I1026 14:00:43.101185   10954 core.go:151] Will not configure cloud provider routes for allocate-node-cidrs: false, configure-cloud-routes: true.
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: W1026 14:00:43.101195   10954 controllermanager.go:508] Skipping "route"
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: I1026 14:00:43.101205   10954 controllermanager.go:501] Starting "nodelifecycle"
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: I1026 14:00:43.101274   10954 disruption.go:288] Starting disruption controller
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: I1026 14:00:43.101290   10954 controller_utils.go:1027] Waiting for caches to sync for disruption controller
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: I1026 14:00:43.349290   10954 node_lifecycle_controller.go:272] Sending events to api server.
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: I1026 14:00:43.349443   10954 node_lifecycle_controller.go:312] Controller is using taint based evictions.
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: I1026 14:00:43.349481   10954 taint_manager.go:175] Sending events to api server.
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: I1026 14:00:43.349526   10954 node_lifecycle_controller.go:378] Controller will taint node by condition.
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: I1026 14:00:43.349564   10954 controllermanager.go:516] Started "nodelifecycle"
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: I1026 14:00:43.349576   10954 controllermanager.go:501] Starting "attachdetach"
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: I1026 14:00:43.349608   10954 node_lifecycle_controller.go:423] Starting node controller
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: I1026 14:00:43.349617   10954 controller_utils.go:1027] Waiting for caches to sync for taint controller
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: I1026 14:00:43.744056   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/aws-ebs"
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: I1026 14:00:43.744101   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/gce-pd"
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: I1026 14:00:43.744115   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/cinder"
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: I1026 14:00:43.744135   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/portworx-volume"
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: I1026 14:00:43.744152   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/vsphere-volume"
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: I1026 14:00:43.744163   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/azure-disk"
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: I1026 14:00:43.744173   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/photon-pd"
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: I1026 14:00:43.744346   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/scaleio"
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: I1026 14:00:43.744368   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/storageos"
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: I1026 14:00:43.744385   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/fc"
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: I1026 14:00:43.744405   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/iscsi"
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: I1026 14:00:43.744415   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/rbd"
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: I1026 14:00:43.744427   10954 plugins.go:547] Loaded volume plugin "kubernetes.io/csi"
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: I1026 14:00:43.745872   10954 controllermanager.go:516] Started "attachdetach"
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: I1026 14:00:43.745889   10954 controllermanager.go:501] Starting "pvc-protection"
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: I1026 14:00:43.747084   10954 attach_detach_controller.go:315] Starting attach detach controller
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: I1026 14:00:43.747109   10954 controller_utils.go:1027] Waiting for caches to sync for attach detach controller
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: I1026 14:00:43.995725   10954 controllermanager.go:516] Started "pvc-protection"
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: I1026 14:00:43.995761   10954 controllermanager.go:501] Starting "resourcequota"
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: I1026 14:00:43.995798   10954 pvc_protection_controller.go:99] Starting PVC protection controller
Oct 26 14:00:43 ctl01 kube-controller-manager[10954]: I1026 14:00:43.995809   10954 controller_utils.go:1027] Waiting for caches to sync for PVC protection controller
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.257106   10954 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for rolebindings.rbac.authorization.k8s.io
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.257169   10954 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for events.events.k8s.io
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.257198   10954 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for horizontalpodautoscalers.autoscaling
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.257244   10954 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for controllerrevisions.apps
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.257272   10954 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for poddisruptionbudgets.policy
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.257512   10954 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for roles.rbac.authorization.k8s.io
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.257548   10954 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for limitranges
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.257579   10954 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for ingresses.extensions
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.257666   10954 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for endpoints
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.257792   10954 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for jobs.batch
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.257910   10954 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for statefulsets.apps
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.257964   10954 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for networkpolicies.networking.k8s.io
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.258061   10954 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for leases.coordination.k8s.io
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.258201   10954 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for serviceaccounts
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.258302   10954 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for replicasets.extensions
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.258337   10954 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for cronjobs.batch
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.258408   10954 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for deployments.apps
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.258472   10954 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for replicasets.apps
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.258552   10954 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for daemonsets.apps
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.258609   10954 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for podtemplates
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.258683   10954 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for deployments.extensions
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.258716   10954 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for daemonsets.extensions
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: E1026 14:00:44.258815   10954 resource_quota_controller.go:171] initial monitor sync has error: couldn't start monitor for resource "extensions/v1beta1, Resource=networkpolicies": unable to monitor quota for resource "extensions/v1beta1, Resource=networkpolicies"
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.258847   10954 controllermanager.go:516] Started "resourcequota"
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.258860   10954 controllermanager.go:501] Starting "job"
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.258947   10954 resource_quota_controller.go:276] Starting resource quota controller
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.258961   10954 controller_utils.go:1027] Waiting for caches to sync for resource quota controller
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.258976   10954 resource_quota_monitor.go:301] QuotaMonitor running
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.495702   10954 controllermanager.go:516] Started "job"
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.495738   10954 controllermanager.go:501] Starting "statefulset"
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.495791   10954 job_controller.go:143] Starting job controller
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.495802   10954 controller_utils.go:1027] Waiting for caches to sync for job controller
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.745076   10954 controllermanager.go:516] Started "statefulset"
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.745111   10954 controllermanager.go:501] Starting "endpoint"
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.745151   10954 stateful_set.go:151] Starting stateful set controller
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.745162   10954 controller_utils.go:1027] Waiting for caches to sync for stateful set controller
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.995732   10954 controllermanager.go:516] Started "endpoint"
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.995774   10954 endpoints_controller.go:160] Starting endpoint controller
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.995798   10954 controller_utils.go:1027] Waiting for caches to sync for endpoint controller
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.995776   10954 controllermanager.go:501] Starting "ttl-after-finished"
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: W1026 14:00:44.995819   10954 controllermanager.go:508] Skipping "ttl-after-finished"
Oct 26 14:00:44 ctl01 kube-controller-manager[10954]: I1026 14:00:44.995828   10954 controllermanager.go:501] Starting "replicationcontroller"
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.251729   10954 controllermanager.go:516] Started "replicationcontroller"
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.251806   10954 controllermanager.go:501] Starting "csrapproving"
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.251778   10954 replica_set.go:182] Starting replicationcontroller controller
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.251842   10954 controller_utils.go:1027] Waiting for caches to sync for ReplicationController controller
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.392045   10954 controllermanager.go:516] Started "csrapproving"
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.398471   10954 certificate_controller.go:113] Starting certificate controller
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.398508   10954 controller_utils.go:1027] Waiting for caches to sync for certificate controller
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.418022   10954 controller_utils.go:1027] Waiting for caches to sync for garbage collector controller
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: W1026 14:00:45.430326   10954 actual_state_of_world.go:491] Failed to update statusUpdateNeeded field in actual state of world: Failed to set statusUpdateNeeded to needed true, because nodeName="ctl01" does not exist
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: W1026 14:00:45.430458   10954 actual_state_of_world.go:491] Failed to update statusUpdateNeeded field in actual state of world: Failed to set statusUpdateNeeded to needed true, because nodeName="cmp002" does not exist
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: W1026 14:00:45.430473   10954 actual_state_of_world.go:491] Failed to update statusUpdateNeeded field in actual state of world: Failed to set statusUpdateNeeded to needed true, because nodeName="cmp001" does not exist
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.445376   10954 controller_utils.go:1034] Caches are synced for stateful set controller
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.446030   10954 controller_utils.go:1034] Caches are synced for TTL controller
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.446914   10954 controller_utils.go:1034] Caches are synced for ReplicaSet controller
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.447759   10954 controller_utils.go:1034] Caches are synced for attach detach controller
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.448838   10954 controller_utils.go:1034] Caches are synced for GC controller
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.449728   10954 controller_utils.go:1034] Caches are synced for taint controller
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.449767   10954 node_lifecycle_controller.go:665] Controller observed a new Node: "cmp002"
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.449789   10954 controller_utils.go:212] Recording Registered Node cmp002 in Controller event message for node cmp002
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.449808   10954 node_lifecycle_controller.go:1222] Initializing eviction metric for zone:
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.449827   10954 node_lifecycle_controller.go:665] Controller observed a new Node: "cmp001"
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.449835   10954 controller_utils.go:212] Recording Registered Node cmp001 in Controller event message for node cmp001
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.449846   10954 node_lifecycle_controller.go:665] Controller observed a new Node: "ctl01"
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.449853   10954 controller_utils.go:212] Recording Registered Node ctl01 in Controller event message for node ctl01
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: W1026 14:00:45.449876   10954 node_lifecycle_controller.go:895] Missing timestamp for Node cmp002. Assuming now as a timestamp.
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: W1026 14:00:45.449900   10954 node_lifecycle_controller.go:895] Missing timestamp for Node cmp001. Assuming now as a timestamp.
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: W1026 14:00:45.449923   10954 node_lifecycle_controller.go:895] Missing timestamp for Node ctl01. Assuming now as a timestamp.
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.449946   10954 node_lifecycle_controller.go:1122] Controller detected that zone  is now in state Normal.
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.449982   10954 event.go:221] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"ctl01", UID:"fbc6a29f-f7f8-11e9-8756-525400d144f2", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RegisteredNode' Node ctl01 event: Registered Node ctl01 in Controller
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.450007   10954 taint_manager.go:198] Starting NoExecuteTaintManager
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.450458   10954 event.go:221] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"cmp002", UID:"fbdf0b9b-f7f8-11e9-8756-525400d144f2", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RegisteredNode' Node cmp002 event: Registered Node cmp002 in Controller
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.450481   10954 event.go:221] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"cmp001", UID:"fe15b5ef-f7f8-11e9-8756-525400d144f2", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RegisteredNode' Node cmp001 event: Registered Node cmp001 in Controller
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.452018   10954 controller_utils.go:1034] Caches are synced for ReplicationController controller
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.453333   10954 controller_utils.go:1034] Caches are synced for ClusterRoleAggregator controller
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.454515   10954 ttl_controller.go:271] Changed ttl annotation for node ctl01 to 0 seconds
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.458570   10954 controller_utils.go:1034] Caches are synced for expand controller
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.461963   10954 controller_utils.go:1034] Caches are synced for daemon sets controller
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: E1026 14:00:45.467977   10954 clusterroleaggregation_controller.go:180] view failed with : Operation cannot be fulfilled on clusterroles.rbac.authorization.k8s.io "view": the object has been modified; please apply your changes to the latest version and try again
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: E1026 14:00:45.468329   10954 clusterroleaggregation_controller.go:180] edit failed with : Operation cannot be fulfilled on clusterroles.rbac.authorization.k8s.io "edit": the object has been modified; please apply your changes to the latest version and try again
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: E1026 14:00:45.471014   10954 clusterroleaggregation_controller.go:180] view failed with : Operation cannot be fulfilled on clusterroles.rbac.authorization.k8s.io "view": the object has been modified; please apply your changes to the latest version and try again
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.495926   10954 controller_utils.go:1034] Caches are synced for PVC protection controller
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.495973   10954 controller_utils.go:1034] Caches are synced for endpoint controller
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.495975   10954 controller_utils.go:1034] Caches are synced for job controller
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.496342   10954 controller_utils.go:1034] Caches are synced for PV protection controller
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.497208   10954 controller_utils.go:1034] Caches are synced for deployment controller
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.499016   10954 controller_utils.go:1034] Caches are synced for certificate controller
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.501171   10954 controller_utils.go:1034] Caches are synced for persistent volume controller
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.501687   10954 controller_utils.go:1034] Caches are synced for disruption controller
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.501705   10954 disruption.go:296] Sending events to api server.
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.501907   10954 ttl_controller.go:271] Changed ttl annotation for node cmp002 to 0 seconds
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.502941   10954 ttl_controller.go:271] Changed ttl annotation for node cmp001 to 0 seconds
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.504659   10954 ttl_controller.go:271] Changed ttl annotation for node cmp002 to 0 seconds
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.506065   10954 ttl_controller.go:271] Changed ttl annotation for node cmp001 to 0 seconds
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.510515   10954 controller_utils.go:1034] Caches are synced for service account controller
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.542015   10954 controller_utils.go:1034] Caches are synced for namespace controller
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.659185   10954 controller_utils.go:1034] Caches are synced for resource quota controller
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.692616   10954 controller_utils.go:1034] Caches are synced for HPA controller
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: I1026 14:00:45.748458   10954 resource_quota_controller.go:427] syncing resource quota controller with updated resources from discovery: map[/v1, Resource=pods:{} /v1, Resource=podtemplates:{} extensions/v1beta1, Resource=replicasets:{} apps/v1, Resource=controllerrevisions:{} policy/v1beta1, Resource=poddisruptionbudgets:{} /v1, Resource=events:{} extensions/v1beta1, Resource=networkpolicies:{} apps/v1, Resource=statefulsets:{} apps/v1, Resource=deployments:{} apps/v1, Resource=daemonsets:{} /v1, Resource=resourcequotas:{} /v1, Resource=persistentvolumeclaims:{} extensions/v1beta1, Resource=daemonsets:{} events.k8s.io/v1beta1, Resource=events:{} networking.k8s.io/v1, Resource=networkpolicies:{} rbac.authorization.k8s.io/v1, Resource=rolebindings:{} /v1, Resource=replicationcontrollers:{} /v1, Resource=services:{} /v1, Resource=serviceaccounts:{} /v1, Resource=endpoints:{} apps/v1, Resource=replicasets:{} batch/v1beta1, Resource=cronjobs:{} /v1, Resource=configmaps:{} /v1, Resource=secrets:{} batch/v1, Resource=jobs:{} rbac.authorization.k8s.io/v1, Resource=roles:{} extensions/v1beta1, Resource=ingresses:{} coordination.k8s.io/v1beta1, Resource=leases:{} /v1, Resource=limitranges:{} extensions/v1beta1, Resource=deployments:{} autoscaling/v1, Resource=horizontalpodautoscalers:{}]
Oct 26 14:00:45 ctl01 kube-controller-manager[10954]: E1026 14:00:45.748847   10954 resource_quota_controller.go:437] failed to sync resource monitors: couldn't start monitor for resource "extensions/v1beta1, Resource=networkpolicies": unable to monitor quota for resource "extensions/v1beta1, Resource=networkpolicies"
Oct 26 14:00:46 ctl01 kube-controller-manager[10954]: I1026 14:00:46.018220   10954 controller_utils.go:1034] Caches are synced for garbage collector controller
Oct 26 14:00:46 ctl01 kube-controller-manager[10954]: I1026 14:00:46.018245   10954 garbagecollector.go:245] synced garbage collector
Oct 26 14:00:46 ctl01 kube-controller-manager[10954]: I1026 14:00:46.042736   10954 controller_utils.go:1034] Caches are synced for garbage collector controller
Oct 26 14:00:46 ctl01 kube-controller-manager[10954]: I1026 14:00:46.042756   10954 garbagecollector.go:142] Garbage collector: all resource monitors have synced. Proceeding to collect garbage
Oct 26 14:00:46 ctl01 kubelet[10393]: I1026 14:00:46.126067   10393 setters.go:72] Using node IP: "172.16.10.36"
Oct 26 14:00:47 ctl01 salt-minion[4526]: [INFO    ] Loading fresh modules for state activity
Oct 26 14:00:47 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:47 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:47 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:48 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:48 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/kubeconfig.sh] at time 14:00:48.316135
Oct 26 14:00:48 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/kubernetes/kubeconfig.sh]
Oct 26 14:00:48 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/kubeconfig.sh'
Oct 26 14:00:48 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:48 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:48 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 14:00:48 ctl01 salt-minion[4526]: New file
Oct 26 14:00:48 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/kubeconfig.sh] at time 14:00:48.708087 duration_in_ms=391.952
Oct 26 14:00:48 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/kubeconfig.sh > /etc/kubernetes/admin-kube-config] at time 14:00:48.711077
Oct 26 14:00:48 ctl01 salt-minion[4526]: [INFO    ] Executing state cmd.run for [/etc/kubernetes/kubeconfig.sh > /etc/kubernetes/admin-kube-config]
Oct 26 14:00:48 ctl01 salt-minion[4526]: [INFO    ] Executing command '/etc/kubernetes/kubeconfig.sh > /etc/kubernetes/admin-kube-config' in directory '/root'
Oct 26 14:00:48 ctl01 salt-minion[4526]: [INFO    ] {'pid': 11497, 'retcode': 0, 'stderr': '', 'stdout': ''}
Oct 26 14:00:48 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/kubeconfig.sh > /etc/kubernetes/admin-kube-config] at time 14:00:48.760805 duration_in_ms=49.73
Oct 26 14:00:48 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/addons/namespace.yaml] at time 14:00:48.761398
Oct 26 14:00:48 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/kubernetes/addons/namespace.yaml]
Oct 26 14:00:48 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/kube-addon-manager/namespace.yaml'
Oct 26 14:00:48 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 14:00:48 ctl01 salt-minion[4526]: New file
Oct 26 14:00:48 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/addons/namespace.yaml] at time 14:00:48.791014 duration_in_ms=29.617
Oct 26 14:00:48 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/kubernetes/manifests/kube-addon-manager.yml] at time 14:00:48.791337
Oct 26 14:00:48 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/etc/kubernetes/manifests/kube-addon-manager.yml]
Oct 26 14:00:48 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/manifest/kube-addon-manager.yml'
Oct 26 14:00:48 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:49 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:49 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 14:00:49 ctl01 salt-minion[4526]: New file
Oct 26 14:00:49 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/kubernetes/manifests/kube-addon-manager.yml] at time 14:00:49.160469 duration_in_ms=369.13
Oct 26 14:00:49 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/default/kube-addon-manager] at time 14:00:49.160978
Oct 26 14:00:49 ctl01 salt-minion[4526]: [INFO    ] Executing state file.absent for [/etc/default/kube-addon-manager]
Oct 26 14:00:49 ctl01 salt-minion[4526]: [INFO    ] File /etc/default/kube-addon-manager is not present
Oct 26 14:00:49 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/default/kube-addon-manager] at time 14:00:49.162509 duration_in_ms=1.53
Oct 26 14:00:49 ctl01 salt-minion[4526]: [INFO    ] Running state [/usr/bin/kube-addons.sh] at time 14:00:49.162887
Oct 26 14:00:49 ctl01 salt-minion[4526]: [INFO    ] Executing state file.absent for [/usr/bin/kube-addons.sh]
Oct 26 14:00:49 ctl01 salt-minion[4526]: [INFO    ] File /usr/bin/kube-addons.sh is not present
Oct 26 14:00:49 ctl01 salt-minion[4526]: [INFO    ] Completed state [/usr/bin/kube-addons.sh] at time 14:00:49.164077 duration_in_ms=1.189
Oct 26 14:00:49 ctl01 kubelet[10393]: E1026 14:00:49.170302   10393 file.go:108] Unable to process watch event: can't process config file "/etc/kubernetes/manifests/kube-addon-manager.ymlmsIy3q": open /etc/kubernetes/manifests/kube-addon-manager.ymlmsIy3q: no such file or directory
Oct 26 14:00:49 ctl01 kubelet[10393]: E1026 14:00:49.170329   10393 file.go:108] Unable to process watch event: can't process config file "/etc/kubernetes/manifests/kube-addon-manager.ymlmsIy3q": open /etc/kubernetes/manifests/kube-addon-manager.ymlmsIy3q: no such file or directory
Oct 26 14:00:49 ctl01 kubelet[10393]: I1026 14:00:49.171227   10393 kubelet.go:1908] SyncLoop (ADD, "file"): "kube-addon-manager-ctl01_kube-system(b6e9bf37167122649376fb470187a295)"
Oct 26 14:00:49 ctl01 kubelet[10393]: I1026 14:00:49.173757   10393 kubelet.go:1918] SyncLoop (REMOVE, "file"): "kube-addon-manager-ctl01_kube-system(b6e9bf37167122649376fb470187a295)"
Oct 26 14:00:49 ctl01 kubelet[10393]: I1026 14:00:49.173813   10393 kubelet.go:2120] Failed to delete pod "kube-addon-manager-ctl01_kube-system(b6e9bf37167122649376fb470187a295)", err: pod not found
Oct 26 14:00:49 ctl01 kubelet[10393]: I1026 14:00:49.173838   10393 kubelet.go:1908] SyncLoop (ADD, "file"): "kube-addon-manager-ctl01_kube-system(b6e9bf37167122649376fb470187a295)"
Oct 26 14:00:49 ctl01 kubelet[10393]: I1026 14:00:49.184743   10393 kubelet.go:1908] SyncLoop (ADD, "api"): "kube-addon-manager-ctl01_kube-system(03a8d15c-f7f9-11e9-8756-525400d144f2)"
Oct 26 14:00:49 ctl01 kubelet[10393]: I1026 14:00:49.202487   10393 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "addons" (UniqueName: "kubernetes.io/host-path/b6e9bf37167122649376fb470187a295-addons") pod "kube-addon-manager-ctl01" (UID: "b6e9bf37167122649376fb470187a295")
Oct 26 14:00:49 ctl01 kubelet[10393]: I1026 14:00:49.202534   10393 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "varlog" (UniqueName: "kubernetes.io/host-path/b6e9bf37167122649376fb470187a295-varlog") pod "kube-addon-manager-ctl01" (UID: "b6e9bf37167122649376fb470187a295")
Oct 26 14:00:49 ctl01 kubelet[10393]: I1026 14:00:49.302919   10393 reconciler.go:252] operationExecutor.MountVolume started for volume "addons" (UniqueName: "kubernetes.io/host-path/b6e9bf37167122649376fb470187a295-addons") pod "kube-addon-manager-ctl01" (UID: "b6e9bf37167122649376fb470187a295")
Oct 26 14:00:49 ctl01 kubelet[10393]: I1026 14:00:49.303011   10393 reconciler.go:252] operationExecutor.MountVolume started for volume "varlog" (UniqueName: "kubernetes.io/host-path/b6e9bf37167122649376fb470187a295-varlog") pod "kube-addon-manager-ctl01" (UID: "b6e9bf37167122649376fb470187a295")
Oct 26 14:00:49 ctl01 kubelet[10393]: I1026 14:00:49.303095   10393 operation_generator.go:571] MountVolume.SetUp succeeded for volume "addons" (UniqueName: "kubernetes.io/host-path/b6e9bf37167122649376fb470187a295-addons") pod "kube-addon-manager-ctl01" (UID: "b6e9bf37167122649376fb470187a295")
Oct 26 14:00:49 ctl01 kubelet[10393]: I1026 14:00:49.303115   10393 operation_generator.go:571] MountVolume.SetUp succeeded for volume "varlog" (UniqueName: "kubernetes.io/host-path/b6e9bf37167122649376fb470187a295-varlog") pod "kube-addon-manager-ctl01" (UID: "b6e9bf37167122649376fb470187a295")
Oct 26 14:00:49 ctl01 kubelet[10393]: I1026 14:00:49.484949   10393 kuberuntime_manager.go:397] No sandbox for pod "kube-addon-manager-ctl01_kube-system(b6e9bf37167122649376fb470187a295)" can be found. Need to start a new one
Oct 26 14:00:49 ctl01 containerd[6508]: time="2019-10-26T14:00:49.488106586Z" level=info msg="RunPodsandbox for &PodSandboxMetadata{Name:kube-addon-manager-ctl01,Uid:b6e9bf37167122649376fb470187a295,Namespace:kube-system,Attempt:0,}"
Oct 26 14:00:49 ctl01 salt-minion[4526]: [INFO    ] Running state [kube-addon-manager] at time 14:00:49.618084
Oct 26 14:00:49 ctl01 salt-minion[4526]: [INFO    ] Executing state service.dead for [kube-addon-manager]
Oct 26 14:00:49 ctl01 salt-minion[4526]: [INFO    ] Executing command ['systemctl', 'status', 'kube-addon-manager.service', '-n', '0'] in directory '/root'
Oct 26 14:00:49 ctl01 salt-minion[4526]: [INFO    ] The named service kube-addon-manager is not available
Oct 26 14:00:49 ctl01 salt-minion[4526]: [INFO    ] Completed state [kube-addon-manager] at time 14:00:49.652660 duration_in_ms=34.576
Oct 26 14:00:49 ctl01 salt-minion[4526]: [INFO    ] Running state [/etc/systemd/system/kube-addon-manager.service] at time 14:00:49.653175
Oct 26 14:00:49 ctl01 salt-minion[4526]: [INFO    ] Executing state file.absent for [/etc/systemd/system/kube-addon-manager.service]
Oct 26 14:00:49 ctl01 salt-minion[4526]: [INFO    ] File /etc/systemd/system/kube-addon-manager.service is not present
Oct 26 14:00:49 ctl01 salt-minion[4526]: [INFO    ] Completed state [/etc/systemd/system/kube-addon-manager.service] at time 14:00:49.654536 duration_in_ms=1.361
Oct 26 14:00:49 ctl01 salt-minion[4526]: [INFO    ] Running state [/srv/kubernetes/conformance.yml] at time 14:00:49.654927
Oct 26 14:00:49 ctl01 salt-minion[4526]: [INFO    ] Executing state file.managed for [/srv/kubernetes/conformance.yml]
Oct 26 14:00:49 ctl01 salt-minion[4526]: [INFO    ] Fetching file from saltenv 'base', ** done ** 'kubernetes/files/conformance/conformance.yml'
Oct 26 14:00:49 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/-.*//g' -e 's/v//g' -e 's/Kubernetes //g' | awk -F'.' '{print $1 "." $2}'' in directory '/root'
Oct 26 14:00:49 ctl01 salt-minion[4526]: [INFO    ] Executing command '(hyperkube --version kubelet 2> /dev/null || echo '0.0') | sed -e 's/+.*//g' -e 's/v//g' -e 's/Kubernetes //g'' in directory '/root'
Oct 26 14:00:50 ctl01 salt-minion[4526]: [INFO    ] File changed:
Oct 26 14:00:50 ctl01 salt-minion[4526]: New file
Oct 26 14:00:50 ctl01 salt-minion[4526]: [INFO    ] Completed state [/srv/kubernetes/conformance.yml] at time 14:00:50.010148 duration_in_ms=355.22
Oct 26 14:00:50 ctl01 salt-minion[4526]: [INFO    ] Returning information for job: 20191026140041088885
Oct 26 14:00:50 ctl01 salt-minion[4526]: [INFO    ] User sudo_ubuntu Executing command cp.push with jid 20191026140050732566
Oct 26 14:00:50 ctl01 salt-minion[4526]: [INFO    ] Starting a new job with PID 11586
Oct 26 14:00:50 ctl01 salt-minion[4526]: [INFO    ] Returning information for job: 20191026140050732566
Oct 26 14:00:50 ctl01 containerd[6508]: time="2019-10-26T14:00:50.926378142Z" level=info msg="ImageCreate event &ImageCreate{Name:docker-prod-local.artifactory.mirantis.com/mirantis/kubernetes/pause-amd64:v1.13.5-3,Labels:map[string]string{},}"
Oct 26 14:00:50 ctl01 containerd[6508]: time="2019-10-26T14:00:50.933019237Z" level=info msg="ImageCreate event &ImageCreate{Name:sha256:478b5c5586708a45bfb7aab46b7a298c4eb65c3442311e7868f49ba269f358a4,Labels:map[string]string{io.cri-containerd.image: managed,},}"
Oct 26 14:00:50 ctl01 containerd[6508]: time="2019-10-26T14:00:50.933984988Z" level=info msg="ImageUpdate event &ImageUpdate{Name:docker-prod-local.artifactory.mirantis.com/mirantis/kubernetes/pause-amd64:v1.13.5-3,Labels:map[string]string{io.cri-containerd.image: managed,},}"
Oct 26 14:00:50 ctl01 containerd[6508]: time="2019-10-26T14:00:50.973407923Z" level=info msg="ImageUpdate event &ImageUpdate{Name:sha256:478b5c5586708a45bfb7aab46b7a298c4eb65c3442311e7868f49ba269f358a4,Labels:map[string]string{io.cri-containerd.image: managed,},}"
Oct 26 14:00:50 ctl01 containerd[6508]: time="2019-10-26T14:00:50.975089040Z" level=info msg="ImageUpdate event &ImageUpdate{Name:docker-prod-local.artifactory.mirantis.com/mirantis/kubernetes/pause-amd64:v1.13.5-3,Labels:map[string]string{io.cri-containerd.image: managed,},}"
Oct 26 14:00:50 ctl01 containerd[6508]: time="2019-10-26T14:00:50.976716806Z" level=info msg="ImageCreate event &ImageCreate{Name:docker-prod-local.artifactory.mirantis.com/mirantis/kubernetes/pause-amd64@sha256:9e2b9d9d64bab9f4d80790ec6c6fe09cdb5714d43bf23357e0ed0d0ab512fffd,Labels:map[string]string{io.cri-containerd.image: managed,},}"
Oct 26 14:00:50 ctl01 containerd[6508]: time="2019-10-26T14:00:50.992074157Z" level=info msg="shim containerd-shim started" address="/containerd-shim/k8s.io/0356dfeae9fd97d18d6ab71d1acea9a2bbeb70b01e9810b821e2f525e1e83109/shim.sock" debug=false pid=11591
Oct 26 14:00:51 ctl01 salt-minion[4526]: [INFO    ] User sudo_ubuntu Executing command cp.push_dir with jid 20191026140051450558
Oct 26 14:00:51 ctl01 salt-minion[4526]: [INFO    ] Starting a new job with PID 11642
Oct 26 14:00:51 ctl01 containerd[6508]: time="2019-10-26T14:00:51.491293104Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:kube-addon-manager-ctl01,Uid:b6e9bf37167122649376fb470187a295,Namespace:kube-system,Attempt:0,} returns sandbox id "0356dfeae9fd97d18d6ab71d1acea9a2bbeb70b01e9810b821e2f525e1e83109""
Oct 26 14:00:51 ctl01 kubelet[10393]: I1026 14:00:51.492545   10393 provider.go:116] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 26 14:00:51 ctl01 containerd[6508]: time="2019-10-26T14:00:51.492857600Z" level=info msg="PullImage "k8s.gcr.io/kube-addon-manager:v8.9""
Oct 26 14:00:51 ctl01 kubelet[10393]: I1026 14:00:51.521984   10393 kubelet.go:1953] SyncLoop (PLEG): "kube-addon-manager-ctl01_kube-system(b6e9bf37167122649376fb470187a295)", event: &pleg.PodLifecycleEvent{ID:"b6e9bf37167122649376fb470187a295", Type:"ContainerStarted", Data:"0356dfeae9fd97d18d6ab71d1acea9a2bbeb70b01e9810b821e2f525e1e83109"}
Oct 26 14:00:51 ctl01 salt-minion[4526]: [INFO    ] Returning information for job: 20191026140051450558
Oct 26 14:00:52 ctl01 containerd[6508]: time="2019-10-26T14:00:52.918838708Z" level=info msg="ImageCreate event &ImageCreate{Name:k8s.gcr.io/kube-addon-manager:v8.9,Labels:map[string]string{},}"
Oct 26 14:00:52 ctl01 containerd[6508]: time="2019-10-26T14:00:52.926972009Z" level=info msg="ImageCreate event &ImageCreate{Name:sha256:13da3ecfc10d338d91f3fecf494fac162c704fc284006ec289d660ec7cbba3e4,Labels:map[string]string{io.cri-containerd.image: managed,},}"
Oct 26 14:00:52 ctl01 containerd[6508]: time="2019-10-26T14:00:52.927832609Z" level=info msg="ImageUpdate event &ImageUpdate{Name:k8s.gcr.io/kube-addon-manager:v8.9,Labels:map[string]string{io.cri-containerd.image: managed,},}"
Oct 26 14:00:55 ctl01 containerd[6508]: time="2019-10-26T14:00:55.676312457Z" level=info msg="ImageUpdate event &ImageUpdate{Name:sha256:13da3ecfc10d338d91f3fecf494fac162c704fc284006ec289d660ec7cbba3e4,Labels:map[string]string{io.cri-containerd.image: managed,},}"
Oct 26 14:00:55 ctl01 containerd[6508]: time="2019-10-26T14:00:55.678502647Z" level=info msg="ImageUpdate event &ImageUpdate{Name:k8s.gcr.io/kube-addon-manager:v8.9,Labels:map[string]string{io.cri-containerd.image: managed,},}"
Oct 26 14:00:55 ctl01 containerd[6508]: time="2019-10-26T14:00:55.680344746Z" level=info msg="ImageCreate event &ImageCreate{Name:k8s.gcr.io/kube-addon-manager@sha256:2fd1daf3d3cf0e94a753f2263b60dbb0d42b107b5cde0c75ee3fc5c830e016e4,Labels:map[string]string{io.cri-containerd.image: managed,},}"
Oct 26 14:00:55 ctl01 containerd[6508]: time="2019-10-26T14:00:55.680687786Z" level=info msg="PullImage "k8s.gcr.io/kube-addon-manager:v8.9" returns image reference "sha256:13da3ecfc10d338d91f3fecf494fac162c704fc284006ec289d660ec7cbba3e4""
Oct 26 14:00:55 ctl01 containerd[6508]: time="2019-10-26T14:00:55.682487491Z" level=info msg="CreateContainer within sandbox "0356dfeae9fd97d18d6ab71d1acea9a2bbeb70b01e9810b821e2f525e1e83109" for container &ContainerMetadata{Name:kube-addon-manager,Attempt:0,}"
Oct 26 14:00:55 ctl01 kernel: [  253.505591] audit: type=1400 audit(1572098455.815:16): apparmor="STATUS" operation="profile_load" profile="unconfined" name="cri-containerd.apparmor.d" pid=11717 comm="apparmor_parser"
Oct 26 14:00:55 ctl01 containerd[6508]: time="2019-10-26T14:00:55.825324907Z" level=info msg="CreateContainer within sandbox "0356dfeae9fd97d18d6ab71d1acea9a2bbeb70b01e9810b821e2f525e1e83109" for &ContainerMetadata{Name:kube-addon-manager,Attempt:0,} returns container id "0cfd9b060ad17356544849dee5a76a1fa8bbcec7b5760936d447442ebcd5709b""
Oct 26 14:00:55 ctl01 containerd[6508]: time="2019-10-26T14:00:55.826280261Z" level=info msg="StartContainer for "0cfd9b060ad17356544849dee5a76a1fa8bbcec7b5760936d447442ebcd5709b""
Oct 26 14:00:55 ctl01 containerd[6508]: time="2019-10-26T14:00:55.827167621Z" level=info msg="shim containerd-shim started" address="/containerd-shim/k8s.io/0cfd9b060ad17356544849dee5a76a1fa8bbcec7b5760936d447442ebcd5709b/shim.sock" debug=false pid=11718
Oct 26 14:00:56 ctl01 kubelet[10393]: I1026 14:00:56.135101   10393 setters.go:72] Using node IP: "172.16.10.36"
Oct 26 14:00:56 ctl01 containerd[6508]: time="2019-10-26T14:00:56.218094198Z" level=info msg="Finish piping stderr of container "0cfd9b060ad17356544849dee5a76a1fa8bbcec7b5760936d447442ebcd5709b""
Oct 26 14:00:56 ctl01 containerd[6508]: time="2019-10-26T14:00:56.218108365Z" level=info msg="Finish piping stdout of container "0cfd9b060ad17356544849dee5a76a1fa8bbcec7b5760936d447442ebcd5709b""
Oct 26 14:00:56 ctl01 containerd[6508]: time="2019-10-26T14:00:56.227346074Z" level=info msg="StartContainer for "0cfd9b060ad17356544849dee5a76a1fa8bbcec7b5760936d447442ebcd5709b" returns successfully"
Oct 26 14:00:56 ctl01 containerd[6508]: time="2019-10-26T14:00:56.230515696Z" level=info msg="CreateContainer within sandbox "0356dfeae9fd97d18d6ab71d1acea9a2bbeb70b01e9810b821e2f525e1e83109" for container &ContainerMetadata{Name:kube-addon-manager,Attempt:0,}"
Oct 26 14:00:56 ctl01 containerd[6508]: time="2019-10-26T14:00:56.239760786Z" level=error msg="CreateContainer within sandbox "0356dfeae9fd97d18d6ab71d1acea9a2bbeb70b01e9810b821e2f525e1e83109" for &ContainerMetadata{Name:kube-addon-manager,Attempt:0,} failed" error="failed to reserve container name "kube-addon-manager_kube-addon-manager-ctl01_kube-system_b6e9bf37167122649376fb470187a295_0": name "kube-addon-manager_kube-addon-manager-ctl01_kube-system_b6e9bf37167122649376fb470187a295_0" is reserved for "0cfd9b060ad17356544849dee5a76a1fa8bbcec7b5760936d447442ebcd5709b""
Oct 26 14:00:56 ctl01 kubelet[10393]: E1026 14:00:56.240619   10393 remote_runtime.go:191] CreateContainer in sandbox "0356dfeae9fd97d18d6ab71d1acea9a2bbeb70b01e9810b821e2f525e1e83109" from runtime service failed: rpc error: code = Unknown desc = failed to reserve container name "kube-addon-manager_kube-addon-manager-ctl01_kube-system_b6e9bf37167122649376fb470187a295_0": name "kube-addon-manager_kube-addon-manager-ctl01_kube-system_b6e9bf37167122649376fb470187a295_0" is reserved for "0cfd9b060ad17356544849dee5a76a1fa8bbcec7b5760936d447442ebcd5709b"
Oct 26 14:00:56 ctl01 kubelet[10393]: E1026 14:00:56.240723   10393 kuberuntime_manager.go:749] container start failed: CreateContainerError: failed to reserve container name "kube-addon-manager_kube-addon-manager-ctl01_kube-system_b6e9bf37167122649376fb470187a295_0": name "kube-addon-manager_kube-addon-manager-ctl01_kube-system_b6e9bf37167122649376fb470187a295_0" is reserved for "0cfd9b060ad17356544849dee5a76a1fa8bbcec7b5760936d447442ebcd5709b"
Oct 26 14:00:56 ctl01 kubelet[10393]: E1026 14:00:56.240795   10393 pod_workers.go:190] Error syncing pod b6e9bf37167122649376fb470187a295 ("kube-addon-manager-ctl01_kube-system(b6e9bf37167122649376fb470187a295)"), skipping: failed to "StartContainer" for "kube-addon-manager" with CreateContainerError: "failed to reserve container name \"kube-addon-manager_kube-addon-manager-ctl01_kube-system_b6e9bf37167122649376fb470187a295_0\": name \"kube-addon-manager_kube-addon-manager-ctl01_kube-system_b6e9bf37167122649376fb470187a295_0\" is reserved for \"0cfd9b060ad17356544849dee5a76a1fa8bbcec7b5760936d447442ebcd5709b\""
Oct 26 14:00:56 ctl01 kubelet[10393]: I1026 14:00:56.531004   10393 kubelet.go:1953] SyncLoop (PLEG): "kube-addon-manager-ctl01_kube-system(b6e9bf37167122649376fb470187a295)", event: &pleg.PodLifecycleEvent{ID:"b6e9bf37167122649376fb470187a295", Type:"ContainerStarted", Data:"0cfd9b060ad17356544849dee5a76a1fa8bbcec7b5760936d447442ebcd5709b"}
Oct 26 14:00:56 ctl01 kube-apiserver[10762]: I1026 14:00:56.666267   10762 controller.go:608] quota admission added evaluator for: namespaces
Oct 26 14:00:57 ctl01 kube-apiserver[10762]: I1026 14:00:57.707361   10762 controller.go:608] quota admission added evaluator for: deployments.extensions
Oct 26 14:00:57 ctl01 kube-apiserver[10762]: I1026 14:00:57.719679   10762 controller.go:608] quota admission added evaluator for: replicasets.apps
Oct 26 14:00:57 ctl01 kube-controller-manager[10954]: I1026 14:00:57.721741   10954 replica_set.go:477] Too few replicas for ReplicaSet kube-system/calico-kube-controllers-996f9b774, need 1, creating 1
Oct 26 14:00:57 ctl01 kube-controller-manager[10954]: I1026 14:00:57.722262   10954 event.go:221] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"kube-system", Name:"calico-kube-controllers", UID:"08bd995b-f7f9-11e9-8756-525400d144f2", APIVersion:"apps/v1", ResourceVersion:"376", FieldPath:""}): type: 'Normal' reason: 'ScalingReplicaSet' Scaled up replica set calico-kube-controllers-996f9b774 to 1
Oct 26 14:00:57 ctl01 kube-controller-manager[10954]: I1026 14:00:57.743247   10954 deployment_controller.go:484] Error syncing deployment kube-system/calico-kube-controllers: Operation cannot be fulfilled on deployments.apps "calico-kube-controllers": the object has been modified; please apply your changes to the latest version and try again
Oct 26 14:00:57 ctl01 kube-controller-manager[10954]: I1026 14:00:57.783393   10954 replica_set.go:477] Too few replicas for ReplicaSet kube-system/coredns-7f8f94c97b, need 2, creating 2
Oct 26 14:00:57 ctl01 kube-controller-manager[10954]: I1026 14:00:57.783517   10954 event.go:221] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"kube-system", Name:"coredns", UID:"08c83e76-f7f9-11e9-8756-525400d144f2", APIVersion:"apps/v1", ResourceVersion:"389", FieldPath:""}): type: 'Normal' reason: 'ScalingReplicaSet' Scaled up replica set coredns-7f8f94c97b to 2
Oct 26 14:00:57 ctl01 kube-controller-manager[10954]: I1026 14:00:57.788355   10954 deployment_controller.go:484] Error syncing deployment kube-system/coredns: Operation cannot be fulfilled on deployments.apps "coredns": the object has been modified; please apply your changes to the latest version and try again
Oct 26 14:00:57 ctl01 kube-proxy[8230]: I1026 14:00:57.819353    8230 service.go:309] Adding new service port "kube-system/coredns:dns" at 10.254.0.10:53/UDP
Oct 26 14:00:57 ctl01 kube-proxy[8230]: I1026 14:00:57.819397    8230 service.go:309] Adding new service port "kube-system/coredns:dns-tcp" at 10.254.0.10:53/TCP
Oct 26 14:00:57 ctl01 kube-apiserver[10762]: I1026 14:00:57.829335   10762 controller.go:608] quota admission added evaluator for: daemonsets.extensions
Oct 26 14:00:57 ctl01 kube-apiserver[10762]: I1026 14:00:57.846119   10762 controller.go:608] quota admission added evaluator for: controllerrevisions.apps
Oct 26 14:00:57 ctl01 kube-controller-manager[10954]: I1026 14:00:57.856847   10954 event.go:221] Event(v1.ObjectReference{Kind:"DaemonSet", Namespace:"netchecker", Name:"netchecker-agent", UID:"08d0381f-f7f9-11e9-8756-525400d144f2", APIVersion:"apps/v1", ResourceVersion:"403", FieldPath:""}): type: 'Normal' reason: 'SuccessfulCreate' Created pod: netchecker-agent-8g7fl
Oct 26 14:00:57 ctl01 kube-controller-manager[10954]: I1026 14:00:57.866049   10954 event.go:221] Event(v1.ObjectReference{Kind:"DaemonSet", Namespace:"netchecker", Name:"netchecker-agent", UID:"08d0381f-f7f9-11e9-8756-525400d144f2", APIVersion:"apps/v1", ResourceVersion:"403", FieldPath:""}): type: 'Normal' reason: 'SuccessfulCreate' Created pod: netchecker-agent-5mrgs
Oct 26 14:00:57 ctl01 kube-controller-manager[10954]: I1026 14:00:57.866170   10954 event.go:221] Event(v1.ObjectReference{Kind:"DaemonSet", Namespace:"netchecker", Name:"netchecker-agent", UID:"08d0381f-f7f9-11e9-8756-525400d144f2", APIVersion:"apps/v1", ResourceVersion:"403", FieldPath:""}): type: 'Normal' reason: 'SuccessfulCreate' Created pod: netchecker-agent-zndnv
Oct 26 14:00:57 ctl01 kubelet[10393]: I1026 14:00:57.870987   10393 kubelet.go:1908] SyncLoop (ADD, "api"): "netchecker-agent-5mrgs_netchecker(08d4a4df-f7f9-11e9-8756-525400d144f2)"
Oct 26 14:00:57 ctl01 kube-controller-manager[10954]: I1026 14:00:57.873860   10954 replica_set.go:477] Too few replicas for ReplicaSet netchecker/netchecker-server-7876fb46d4, need 1, creating 1
Oct 26 14:00:57 ctl01 kube-controller-manager[10954]: I1026 14:00:57.875960   10954 event.go:221] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"netchecker", Name:"netchecker-server", UID:"08d5db39-f7f9-11e9-8756-525400d144f2", APIVersion:"apps/v1", ResourceVersion:"413", FieldPath:""}): type: 'Normal' reason: 'ScalingReplicaSet' Scaled up replica set netchecker-server-7876fb46d4 to 1
Oct 26 14:00:57 ctl01 kube-controller-manager[10954]: I1026 14:00:57.883827   10954 replica_set.go:514] Slow-start failure. Skipping creation of 1 pods, decrementing expectations for ReplicaSet netchecker/netchecker-server-7876fb46d4
Oct 26 14:00:57 ctl01 kube-controller-manager[10954]: I1026 14:00:57.883896   10954 event.go:221] Event(v1.ObjectReference{Kind:"ReplicaSet", Namespace:"netchecker", Name:"netchecker-server-7876fb46d4", UID:"08d66706-f7f9-11e9-8756-525400d144f2", APIVersion:"apps/v1", ResourceVersion:"419", FieldPath:""}): type: 'Warning' reason: 'FailedCreate' Error creating: pods "netchecker-server-7876fb46d4-" is forbidden: error looking up service account netchecker/netchecker: serviceaccount "netchecker" not found
Oct 26 14:00:57 ctl01 kube-controller-manager[10954]: E1026 14:00:57.887537   10954 daemon_controller.go:304] netchecker/netchecker-agent failed with : error storing status for daemon set &v1.DaemonSet{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"netchecker-agent", GenerateName:"", Namespace:"netchecker", SelfLink:"/apis/apps/v1/namespaces/netchecker/daemonsets/netchecker-agent", UID:"08d0381f-f7f9-11e9-8756-525400d144f2", ResourceVersion:"403", Generation:1, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63707695257, loc:(*time.Location)(0xaf74780)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"addonmanager.kubernetes.io/mode":"Reconcile", "app":"netchecker-agent"}, Annotations:map[string]string{"deprecated.daemonset.template.generation":"1", "kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"extensions/v1beta1\",\"kind\":\"DaemonSet\",\"metadata\":{\"annotations\":{},\"labels\":{\"addonmanager.kubernetes.io/mode\":\"Reconcile\",\"app\":\"netchecker-agent\"},\"name\":\"netchecker-agent\",\"namespace\":\"netchecker\"},\"spec\":{\"template\":{\"metadata\":{\"annotations\":null,\"labels\":{\"app\":\"netchecker-agent\"},\"name\":\"netchecker-agent\"},\"spec\":{\"containers\":[{\"args\":[\"-v=5\",\"-alsologtostderr=true\",\"-serverendpoint=netchecker:80\",\"-reportinterval=60\",\"-probeurls=http://ipinfo.io\"],\"env\":[{\"name\":\"MY_NODE_NAME\",\"valueFrom\":{\"fieldRef\":{\"fieldPath\":\"spec.nodeName\"}}},{\"name\":\"MY_POD_NAME\",\"valueFrom\":{\"fieldRef\":{\"fieldPath\":\"metadata.name\"}}}],\"image\":\"mirantis/k8s-netchecker-agent:stable\",\"name\":\"netchecker-agent\"}],\"tolerations\":[{\"effect\":\"NoSchedule\",\"key\":\"node-role.kubernetes.io/master\"}]}}}}\n"}, OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1.DaemonSetSpec{Selector:(*v1.LabelSelector)(0xc0022c0d20), Template:v1.PodTemplateSpec{ObjectMeta:v1.ObjectMeta{Name:"netchecker-agent", GenerateName:"",
Oct 26 14:00:57 ctl01 kube-controller-manager[10954]: Namespace:"", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app":"netchecker-agent"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v1.PodSpec{Volumes:[]v1.Volume(nil), InitContainers:[]v1.Container(nil), Containers:[]v1.Container{v1.Container{Name:"netchecker-agent", Image:"mirantis/k8s-netchecker-agent:stable", Command:[]string(nil), Args:[]string{"-v=5", "-alsologtostderr=true", "-serverendpoint=netchecker:80", "-reportinterval=60", "-probeurls=http://ipinfo.io"}, WorkingDir:"", Ports:[]v1.ContainerPort(nil), EnvFrom:[]v1.EnvFromSource(nil), Env:[]v1.EnvVar{v1.EnvVar{Name:"MY_NODE_NAME", Value:"", ValueFrom:(*v1.EnvVarSource)(0xc0022c0d60)}, v1.EnvVar{Name:"MY_POD_NAME", Value:"", ValueFrom:(*v1.EnvVarSource)(0xc0022c0da0)}}, Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList(nil)}, VolumeMounts:[]v1.VolumeMount(nil), VolumeDevices:[]v1.VolumeDevice(nil), LivenessProbe:(*v1.Probe)(nil), ReadinessProbe:(*v1.Probe)(nil), Lifecycle:(*v1.Lifecycle)(nil), TerminationMessagePath:"/dev/termination-log", TerminationMessagePolicy:"File", ImagePullPolicy:"IfNotPresent", SecurityContext:(*v1.SecurityContext)(nil), Stdin:false, StdinOnce:false, TTY:false}}, RestartPolicy:"Always", TerminationGracePeriodSeconds:(*int64)(0xc0022c44e8), ActiveDeadlineSeconds:(*int64)(nil), DNSPolicy:"ClusterFirst", NodeSelector:map[string]string(nil), ServiceAccountName:"", DeprecatedServiceAccount:"", AutomountServiceAccountToken:(*bool)(nil), NodeName:"", HostNetwork:false, HostPID:false, HostIPC:false, ShareProcessNamespace:(*bool)(nil), SecurityContext:(*v1.PodSecurityContext)(0xc0022c8c00), ImagePullSecrets:[]v1.LocalObjectReference(nil), Hostname:"", Subdomain:"", Affinity:(*v1.Affinity)(
Oct 26 14:00:57 ctl01 kube-controller-manager[10954]: nil), SchedulerName:"default-scheduler", Tolerations:[]v1.Toleration{v1.Toleration{Key:"node-role.kubernetes.io/master", Operator:"", Value:"", Effect:"NoSchedule", TolerationSeconds:(*int64)(nil)}}, HostAliases:[]v1.HostAlias(nil), PriorityClassName:"", Priority:(*int32)(nil), DNSConfig:(*v1.PodDNSConfig)(nil), ReadinessGates:[]v1.PodReadinessGate(nil), RuntimeClassName:(*string)(nil), EnableServiceLinks:(*bool)(nil)}}, UpdateStrategy:v1.DaemonSetUpdateStrategy{Type:"OnDelete", RollingUpdate:(*v1.RollingUpdateDaemonSet)(nil)}, MinReadySeconds:0, RevisionHistoryLimit:(*int32)(0xc0022c4518)}, Status:v1.DaemonSetStatus{CurrentNumberScheduled:0, NumberMisscheduled:0, DesiredNumberScheduled:0, NumberReady:0, ObservedGeneration:0, UpdatedNumberScheduled:0, NumberAvailable:0, NumberUnavailable:0, CollisionCount:(*int32)(nil), Conditions:[]v1.DaemonSetCondition(nil)}}: Operation cannot be fulfilled on daemonsets.apps "netchecker-agent": the object has been modified; please apply your changes to the latest version and try again
Oct 26 14:00:57 ctl01 kube-controller-manager[10954]: E1026 14:00:57.887550   10954 replica_set.go:450] Sync "netchecker/netchecker-server-7876fb46d4" failed with pods "netchecker-server-7876fb46d4-" is forbidden: error looking up service account netchecker/netchecker: serviceaccount "netchecker" not found
Oct 26 14:00:57 ctl01 kube-controller-manager[10954]: I1026 14:00:57.888748   10954 replica_set.go:477] Too few replicas for ReplicaSet netchecker/netchecker-server-7876fb46d4, need 1, creating 1
Oct 26 14:00:57 ctl01 kube-controller-manager[10954]: I1026 14:00:57.894033   10954 deployment_controller.go:484] Error syncing deployment netchecker/netchecker-server: Operation cannot be fulfilled on deployments.apps "netchecker-server": the object has been modified; please apply your changes to the latest version and try again
Oct 26 14:00:57 ctl01 kube-controller-manager[10954]: I1026 14:00:57.964253   10954 deployment_controller.go:484] Error syncing deployment netchecker/netchecker-server: Operation cannot be fulfilled on deployments.apps "netchecker-server": the object has been modified; please apply your changes to the latest version and try again
Oct 26 14:00:57 ctl01 kube-proxy[8230]: I1026 14:00:57.975297    8230 service.go:309] Adding new service port "netchecker/netchecker:" at 10.254.174.248:80/TCP
Oct 26 14:00:57 ctl01 kube-proxy[8230]: I1026 14:00:57.993688    8230 proxier.go:1427] Opened local port "nodePort for netchecker/netchecker:" (:30423/tcp)
Oct 26 14:00:58 ctl01 kubelet[10393]: I1026 14:00:58.021541   10393 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "default-token-q7f5k" (UniqueName: "kubernetes.io/secret/08d4a4df-f7f9-11e9-8756-525400d144f2-default-token-q7f5k") pod "netchecker-agent-5mrgs" (UID: "08d4a4df-f7f9-11e9-8756-525400d144f2")
Oct 26 14:00:58 ctl01 kubelet[10393]: I1026 14:00:58.121998   10393 reconciler.go:252] operationExecutor.MountVolume started for volume "default-token-q7f5k" (UniqueName: "kubernetes.io/secret/08d4a4df-f7f9-11e9-8756-525400d144f2-default-token-q7f5k") pod "netchecker-agent-5mrgs" (UID: "08d4a4df-f7f9-11e9-8756-525400d144f2")
Oct 26 14:00:58 ctl01 systemd[1]: Started Kubernetes transient mount for /var/lib/kubelet/pods/08d4a4df-f7f9-11e9-8756-525400d144f2/volumes/kubernetes.io~secret/default-token-q7f5k.
Oct 26 14:00:58 ctl01 kubelet[10393]: I1026 14:00:58.136187   10393 operation_generator.go:571] MountVolume.SetUp succeeded for volume "default-token-q7f5k" (UniqueName: "kubernetes.io/secret/08d4a4df-f7f9-11e9-8756-525400d144f2-default-token-q7f5k") pod "netchecker-agent-5mrgs" (UID: "08d4a4df-f7f9-11e9-8756-525400d144f2")
Oct 26 14:00:58 ctl01 kubelet[10393]: I1026 14:00:58.181172   10393 kuberuntime_manager.go:397] No sandbox for pod "netchecker-agent-5mrgs_netchecker(08d4a4df-f7f9-11e9-8756-525400d144f2)" can be found. Need to start a new one
Oct 26 14:00:58 ctl01 containerd[6508]: time="2019-10-26T14:00:58.182125690Z" level=info msg="RunPodsandbox for &PodSandboxMetadata{Name:netchecker-agent-5mrgs,Uid:08d4a4df-f7f9-11e9-8756-525400d144f2,Namespace:netchecker,Attempt:0,}"
Oct 26 14:00:58 ctl01 containerd[6508]: 2019-10-26 14:00:58.456 [INFO][11976] calico.go 75: Extracted identifiers EndpointIDs=&utils.WEPIdentifiers{Namespace:"netchecker", WEPName:"", WorkloadEndpointIdentifiers:names.WorkloadEndpointIdentifiers{Node:"ctl01", Orchestrator:"k8s", Endpoint:"eth0", Workload:"", Pod:"netchecker-agent-5mrgs", ContainerID:"b875754565fd9f2dcdab462102a962ed536b19fab9397471ec6c682c28883867"}}
Oct 26 14:00:58 ctl01 containerd[6508]: 2019-10-26 14:00:58.519 [INFO][11976] k8s.go 60: Extracted identifiers for CmdAddK8s ContainerID="b875754565fd9f2dcdab462102a962ed536b19fab9397471ec6c682c28883867" Namespace="netchecker" Pod="netchecker-agent-5mrgs" WorkloadEndpoint="ctl01-k8s-netchecker--agent--5mrgs-eth0"
Oct 26 14:00:58 ctl01 containerd[6508]: Calico CNI IPAM request count IPv4=1 IPv6=0
Oct 26 14:00:58 ctl01 containerd[6508]: Calico CNI IPAM handle=calico-k8s-network.b875754565fd9f2dcdab462102a962ed536b19fab9397471ec6c682c28883867
Oct 26 14:00:58 ctl01 containerd[6508]: 2019-10-26 14:00:58.625 [INFO][11991] calico-ipam.go 186: Auto assigning IP ContainerID="b875754565fd9f2dcdab462102a962ed536b19fab9397471ec6c682c28883867" HandleID="calico-k8s-network.b875754565fd9f2dcdab462102a962ed536b19fab9397471ec6c682c28883867" Workload="ctl01-k8s-netchecker--agent--5mrgs-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc420248ee0), Attrs:map[string]string(nil), Hostname:"ctl01", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}}
Oct 26 14:00:58 ctl01 containerd[6508]: 2019-10-26 14:00:58.625 [INFO][11991] ipam.go 70: Auto-assign 1 ipv4, 0 ipv6 addrs for host 'ctl01'
Oct 26 14:00:58 ctl01 containerd[6508]: 2019-10-26 14:00:58.626 [INFO][11991] ipam.go 254: Looking up existing affinities for host handle="calico-k8s-network.b875754565fd9f2dcdab462102a962ed536b19fab9397471ec6c682c28883867" host="ctl01"
Oct 26 14:00:58 ctl01 containerd[6508]: 2019-10-26 14:00:58.627 [INFO][11991] ipam.go 265: Ran out of existing affine blocks for host handle="calico-k8s-network.b875754565fd9f2dcdab462102a962ed536b19fab9397471ec6c682c28883867" host="ctl01"
Oct 26 14:00:58 ctl01 containerd[6508]: 2019-10-26 14:00:58.628 [INFO][11991] ipam.go 324: No more affine blocks, but need to allocate 1 more addresses - allocate another block handle="calico-k8s-network.b875754565fd9f2dcdab462102a962ed536b19fab9397471ec6c682c28883867" host="ctl01"
Oct 26 14:00:58 ctl01 containerd[6508]: 2019-10-26 14:00:58.628 [INFO][11991] ipam.go 328: Looking for an unclaimed block handle="calico-k8s-network.b875754565fd9f2dcdab462102a962ed536b19fab9397471ec6c682c28883867" host="ctl01"
Oct 26 14:00:58 ctl01 containerd[6508]: 2019-10-26 14:00:58.628 [INFO][11991] ipam_block_reader_writer.go 106: Found free block: 192.168.237.0/26
Oct 26 14:00:58 ctl01 containerd[6508]: 2019-10-26 14:00:58.629 [INFO][11991] ipam.go 340: Found unclaimed block host="ctl01" subnet=192.168.237.0/26
Oct 26 14:00:58 ctl01 containerd[6508]: 2019-10-26 14:00:58.629 [INFO][11991] ipam_block_reader_writer.go 122: Trying to create affinity in pending state host="ctl01" subnet=192.168.237.0/26
Oct 26 14:00:58 ctl01 containerd[6508]: 2019-10-26 14:00:58.630 [INFO][11991] ipam_block_reader_writer.go 152: Successfully created pending affinity for block host="ctl01" subnet=192.168.237.0/26
Oct 26 14:00:58 ctl01 containerd[6508]: 2019-10-26 14:00:58.630 [INFO][11991] ipam.go 118: Attempting to load block cidr=192.168.237.0/26 host="ctl01"
Oct 26 14:00:58 ctl01 containerd[6508]: 2019-10-26 14:00:58.630 [INFO][11991] ipam.go 123: The referenced block doesn't exist, trying to create it cidr=192.168.237.0/26 host="ctl01"
Oct 26 14:00:58 ctl01 containerd[6508]: 2019-10-26 14:00:58.631 [INFO][11991] ipam.go 130: Wrote affinity as pending cidr=192.168.237.0/26 host="ctl01"
Oct 26 14:00:58 ctl01 containerd[6508]: 2019-10-26 14:00:58.632 [INFO][11991] ipam.go 139: Attempting to claim the block cidr=192.168.237.0/26 host="ctl01"
Oct 26 14:00:58 ctl01 containerd[6508]: 2019-10-26 14:00:58.632 [INFO][11991] ipam_block_reader_writer.go 175: Attempting to create a new block host="ctl01" subnet=192.168.237.0/26
Oct 26 14:00:58 ctl01 containerd[6508]: 2019-10-26 14:00:58.635 [INFO][11991] ipam_block_reader_writer.go 217: Successfully created block
Oct 26 14:00:58 ctl01 containerd[6508]: 2019-10-26 14:00:58.635 [INFO][11991] ipam_block_reader_writer.go 228: Confirming affinity host="ctl01" subnet=192.168.237.0/26
Oct 26 14:00:58 ctl01 containerd[6508]: 2019-10-26 14:00:58.637 [INFO][11991] ipam_block_reader_writer.go 243: Successfully confirmed affinity host="ctl01" subnet=192.168.237.0/26
Oct 26 14:00:58 ctl01 containerd[6508]: 2019-10-26 14:00:58.637 [INFO][11991] ipam.go 372: Claimed new block &{BlockKey(cidr=192.168.237.0/26) 0xc4201ee900 441 0s} - assigning 1 addresses host="ctl01" subnet=192.168.237.0/26
Oct 26 14:00:58 ctl01 containerd[6508]: 2019-10-26 14:00:58.637 [INFO][11991] ipam.go 677: Attempting to assign 1 addresses from block block=192.168.237.0/26 handle="calico-k8s-network.b875754565fd9f2dcdab462102a962ed536b19fab9397471ec6c682c28883867" host="ctl01"
Oct 26 14:00:58 ctl01 containerd[6508]: 2019-10-26 14:00:58.638 [INFO][11991] ipam.go 1110: Creating new handle: calico-k8s-network.b875754565fd9f2dcdab462102a962ed536b19fab9397471ec6c682c28883867
Oct 26 14:00:58 ctl01 containerd[6508]: 2019-10-26 14:00:58.639 [INFO][11991] ipam.go 700: Writing block in order to claim IPs block=192.168.237.0/26 handle="calico-k8s-network.b875754565fd9f2dcdab462102a962ed536b19fab9397471ec6c682c28883867" host="ctl01"
Oct 26 14:00:58 ctl01 containerd[6508]: 2019-10-26 14:00:58.641 [INFO][11991] ipam.go 710: Successfully claimed IPs: [192.168.237.0] block=192.168.237.0/26 handle="calico-k8s-network.b875754565fd9f2dcdab462102a962ed536b19fab9397471ec6c682c28883867" host="ctl01"
Oct 26 14:00:58 ctl01 containerd[6508]: 2019-10-26 14:00:58.641 [INFO][11991] ipam.go 456: Auto-assigned 1 out of 1 IPv4s: [192.168.237.0] handle="calico-k8s-network.b875754565fd9f2dcdab462102a962ed536b19fab9397471ec6c682c28883867" host="ctl01"
Oct 26 14:00:58 ctl01 containerd[6508]: Calico CNI IPAM assigned addresses IPv4=[192.168.237.0] IPv6=[]
Oct 26 14:00:58 ctl01 containerd[6508]: 2019-10-26 14:00:58.641 [INFO][11991] calico-ipam.go 214: IPAM Result ContainerID="b875754565fd9f2dcdab462102a962ed536b19fab9397471ec6c682c28883867" HandleID="calico-k8s-network.b875754565fd9f2dcdab462102a962ed536b19fab9397471ec6c682c28883867" Workload="ctl01-k8s-netchecker--agent--5mrgs-eth0" result.IPs=[]*current.IPConfig{(*current.IPConfig)(0xc4200927e0)}
Oct 26 14:00:58 ctl01 containerd[6508]: 2019-10-26 14:00:58.642 [INFO][11976] k8s.go 365: Populated endpoint ContainerID="b875754565fd9f2dcdab462102a962ed536b19fab9397471ec6c682c28883867" Namespace="netchecker" Pod="netchecker-agent-5mrgs" WorkloadEndpoint="ctl01-k8s-netchecker--agent--5mrgs-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"ctl01-k8s-netchecker--agent--5mrgs-eth0", GenerateName:"", Namespace:"netchecker", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"ctl01", ContainerID:"", Pod:"netchecker-agent-5mrgs", Endpoint:"eth0", IPNetworks:[]string{"192.168.237.0/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"calico-k8s-network"}, InterfaceName:"", MAC:"", Ports:[]v3.EndpointPort(nil)}}
Oct 26 14:00:58 ctl01 containerd[6508]: Calico CNI using IPs: [192.168.237.0/32]
Oct 26 14:00:58 ctl01 containerd[6508]: 2019-10-26 14:00:58.643 [INFO][11976] network.go 75: Setting the host side veth name to calibd967d734cf ContainerID="b875754565fd9f2dcdab462102a962ed536b19fab9397471ec6c682c28883867" Namespace="netchecker" Pod="netchecker-agent-5mrgs" WorkloadEndpoint="ctl01-k8s-netchecker--agent--5mrgs-eth0"
Oct 26 14:00:58 ctl01 kernel: [  256.335478] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
Oct 26 14:00:58 ctl01 kernel: [  256.335698] IPv6: ADDRCONF(NETDEV_UP): calibd967d734cf: link is not ready
Oct 26 14:00:58 ctl01 kernel: [  256.335708] IPv6: ADDRCONF(NETDEV_CHANGE): calibd967d734cf: link becomes ready
Oct 26 14:00:58 ctl01 kernel: [  256.335745] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Oct 26 14:00:58 ctl01 containerd[6508]: 2019-10-26 14:00:58.649 [INFO][11976] network.go 380: Disabling IPv4 forwarding ContainerID="b875754565fd9f2dcdab462102a962ed536b19fab9397471ec6c682c28883867" Namespace="netchecker" Pod="netchecker-agent-5mrgs" WorkloadEndpoint="ctl01-k8s-netchecker--agent--5mrgs-eth0"
Oct 26 14:00:58 ctl01 containerd[6508]: 2019-10-26 14:00:58.690 [INFO][11976] k8s.go 392: Added Mac, interface name, and active container ID to endpoint ContainerID="b875754565fd9f2dcdab462102a962ed536b19fab9397471ec6c682c28883867" Namespace="netchecker" Pod="netchecker-agent-5mrgs" WorkloadEndpoint="ctl01-k8s-netchecker--agent--5mrgs-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"ctl01-k8s-netchecker--agent--5mrgs-eth0", GenerateName:"", Namespace:"netchecker", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"ctl01", ContainerID:"b875754565fd9f2dcdab462102a962ed536b19fab9397471ec6c682c28883867", Pod:"netchecker-agent-5mrgs", Endpoint:"eth0", IPNetworks:[]string{"192.168.237.0/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"calico-k8s-network"}, InterfaceName:"calibd967d734cf", MAC:"62:f8:25:00:45:ed", Ports:[]v3.EndpointPort(nil)}}
Oct 26 14:00:58 ctl01 containerd[6508]: 2019-10-26 14:00:58.698 [INFO][11976] k8s.go 424: Wrote updated endpoint to datastore ContainerID="b875754565fd9f2dcdab462102a962ed536b19fab9397471ec6c682c28883867" Namespace="netchecker" Pod="netchecker-agent-5mrgs" WorkloadEndpoint="ctl01-k8s-netchecker--agent--5mrgs-eth0"
Oct 26 14:00:58 ctl01 containerd[6508]: Calico CNI creating profile: calico-k8s-network
Oct 26 14:00:58 ctl01 containerd[6508]: 2019-10-26 14:00:58.702 [INFO][11976] calico.go 366: Creating profile ContainerID="b875754565fd9f2dcdab462102a962ed536b19fab9397471ec6c682c28883867" Namespace="netchecker" Pod="netchecker-agent-5mrgs" WorkloadEndpoint="ctl01-k8s-netchecker--agent--5mrgs-" profile=&v3.Profile{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"calico-k8s-network", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v3.ProfileSpec{Ingress:[]v3.Rule{v3.Rule{Action:"Allow", IPVersion:(*int)(nil), Protocol:(*numorstring.Protocol)(nil), ICMP:(*v3.ICMPFields)(nil), NotProtocol:(*numorstring.Protocol)(nil), NotICMP:(*v3.ICMPFields)(nil), Source:v3.EntityRule{Nets:[]string(nil), Selector:"", NamespaceSelector:"", Ports:[]numorstring.Port(nil), NotNets:[]string(nil), NotSelector:"", NotPorts:[]numorstring.Port(nil), ServiceAccounts:(*v3.ServiceAccountMatch)(nil)}, Destination:v3.EntityRule{Nets:[]string(nil), Selector:"", NamespaceSelector:"", Ports:[]numorstring.Port(nil), NotNets:[]string(nil), NotSelector:"", NotPorts:[]numorstring.Port(nil), ServiceAccounts:(*v3.ServiceAccountMatch)(nil)}, HTTP:(*v3.HTTPMatch)(nil)}}, Egress:[]v3.Rule{v3.Rule{Action:"Allow", IPVersion:(*int)(nil), Protocol:(*numorstring.Protocol)(nil), ICMP:(*v3.ICMPFields)(nil), NotProtocol:(*numorstring.Protocol)(nil), NotICMP:(*v3.ICMPFields)(nil), Source:v3.EntityRule{Nets:[]string(nil), Selector:"", NamespaceSelector:"", Ports:[]numorstring.Port(nil), NotNets:[]string(nil), NotSelector:"", NotPorts:[]numorstring.Port(nil), ServiceAccounts:(*v3.ServiceAccountMatch)(nil)}, Destination:v3.EntityRule{Nets:[]string(nil), Selector:"", NamespaceSelector:"", Por
Oct 26 14:00:58 ctl01 containerd[6508]: ts:[]numorstring.Port(nil), NotNets:[]string(nil), NotSelector:"", NotPorts:[]numorstring.Port(nil), ServiceAccounts:(*v3.ServiceAccountMatch)(nil)}, HTTP:(*v3.HTTPMatch)(nil)}}, LabelsToApply:map[string]string{"calico-k8s-network":""}}}
Oct 26 14:00:58 ctl01 containerd[6508]: time="2019-10-26T14:00:58.719544309Z" level=info msg="shim containerd-shim started" address="/containerd-shim/k8s.io/b875754565fd9f2dcdab462102a962ed536b19fab9397471ec6c682c28883867/shim.sock" debug=false pid=12052
Oct 26 14:00:58 ctl01 kube-controller-manager[10954]: I1026 14:00:58.747689   10954 event.go:221] Event(v1.ObjectReference{Kind:"ReplicaSet", Namespace:"kube-system", Name:"calico-kube-controllers-996f9b774", UID:"08bf7c8d-f7f9-11e9-8756-525400d144f2", APIVersion:"apps/v1", ResourceVersion:"377", FieldPath:""}): type: 'Normal' reason: 'SuccessfulCreate' Created pod: calico-kube-controllers-996f9b774-lkk6r
Oct 26 14:00:58 ctl01 kubelet[10393]: I1026 14:00:58.750947   10393 kubelet.go:1908] SyncLoop (ADD, "api"): "calico-kube-controllers-996f9b774-lkk6r_kube-system(095be993-f7f9-11e9-8756-525400d144f2)"
Oct 26 14:00:58 ctl01 kube-controller-manager[10954]: I1026 14:00:58.791767   10954 event.go:221] Event(v1.ObjectReference{Kind:"ReplicaSet", Namespace:"kube-system", Name:"coredns-7f8f94c97b", UID:"08c8d971-f7f9-11e9-8756-525400d144f2", APIVersion:"apps/v1", ResourceVersion:"390", FieldPath:""}): type: 'Normal' reason: 'SuccessfulCreate' Created pod: coredns-7f8f94c97b-mvmhn
Oct 26 14:00:58 ctl01 kube-controller-manager[10954]: I1026 14:00:58.795773   10954 event.go:221] Event(v1.ObjectReference{Kind:"ReplicaSet", Namespace:"kube-system", Name:"coredns-7f8f94c97b", UID:"08c8d971-f7f9-11e9-8756-525400d144f2", APIVersion:"apps/v1", ResourceVersion:"390", FieldPath:""}): type: 'Normal' reason: 'SuccessfulCreate' Created pod: coredns-7f8f94c97b-fkm4q
Oct 26 14:00:58 ctl01 kubelet[10393]: I1026 14:00:58.799044   10393 kubelet.go:1908] SyncLoop (ADD, "api"): "coredns-7f8f94c97b-fkm4q_kube-system(09632cce-f7f9-11e9-8756-525400d144f2)"
Oct 26 14:00:58 ctl01 kubelet[10393]: I1026 14:00:58.823404   10393 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "etcd-certs" (UniqueName: "kubernetes.io/host-path/095be993-f7f9-11e9-8756-525400d144f2-etcd-certs") pod "calico-kube-controllers-996f9b774-lkk6r" (UID: "095be993-f7f9-11e9-8756-525400d144f2")
Oct 26 14:00:58 ctl01 kubelet[10393]: I1026 14:00:58.823464   10393 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "calico-kube-controllers-token-gxwp4" (UniqueName: "kubernetes.io/secret/095be993-f7f9-11e9-8756-525400d144f2-calico-kube-controllers-token-gxwp4") pod "calico-kube-controllers-996f9b774-lkk6r" (UID: "095be993-f7f9-11e9-8756-525400d144f2")
Oct 26 14:00:58 ctl01 kubelet[10393]: I1026 14:00:58.823503   10393 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "config-volume" (UniqueName: "kubernetes.io/configmap/09632cce-f7f9-11e9-8756-525400d144f2-config-volume") pod "coredns-7f8f94c97b-fkm4q" (UID: "09632cce-f7f9-11e9-8756-525400d144f2")
Oct 26 14:00:58 ctl01 kubelet[10393]: I1026 14:00:58.823610   10393 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "coredns-token-8cvdr" (UniqueName: "kubernetes.io/secret/09632cce-f7f9-11e9-8756-525400d144f2-coredns-token-8cvdr") pod "coredns-7f8f94c97b-fkm4q" (UID: "09632cce-f7f9-11e9-8756-525400d144f2")
Oct 26 14:00:58 ctl01 kube-controller-manager[10954]: I1026 14:00:58.895567   10954 event.go:221] Event(v1.ObjectReference{Kind:"ReplicaSet", Namespace:"netchecker", Name:"netchecker-server-7876fb46d4", UID:"08d66706-f7f9-11e9-8756-525400d144f2", APIVersion:"apps/v1", ResourceVersion:"426", FieldPath:""}): type: 'Normal' reason: 'SuccessfulCreate' Created pod: netchecker-server-7876fb46d4-r46ls
Oct 26 14:00:58 ctl01 kubelet[10393]: I1026 14:00:58.899127   10393 kubelet.go:1908] SyncLoop (ADD, "api"): "netchecker-server-7876fb46d4-r46ls_netchecker(09723ce9-f7f9-11e9-8756-525400d144f2)"
Oct 26 14:00:58 ctl01 containerd[6508]: time="2019-10-26T14:00:58.913794844Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:netchecker-agent-5mrgs,Uid:08d4a4df-f7f9-11e9-8756-525400d144f2,Namespace:netchecker,Attempt:0,} returns sandbox id "b875754565fd9f2dcdab462102a962ed536b19fab9397471ec6c682c28883867""
Oct 26 14:00:58 ctl01 containerd[6508]: time="2019-10-26T14:00:58.915313477Z" level=info msg="PullImage "mirantis/k8s-netchecker-agent:stable""
Oct 26 14:00:58 ctl01 kubelet[10393]: I1026 14:00:58.923888   10393 reconciler.go:252] operationExecutor.MountVolume started for volume "etcd-certs" (UniqueName: "kubernetes.io/host-path/095be993-f7f9-11e9-8756-525400d144f2-etcd-certs") pod "calico-kube-controllers-996f9b774-lkk6r" (UID: "095be993-f7f9-11e9-8756-525400d144f2")
Oct 26 14:00:58 ctl01 kubelet[10393]: I1026 14:00:58.923935   10393 reconciler.go:252] operationExecutor.MountVolume started for volume "calico-kube-controllers-token-gxwp4" (UniqueName: "kubernetes.io/secret/095be993-f7f9-11e9-8756-525400d144f2-calico-kube-controllers-token-gxwp4") pod "calico-kube-controllers-996f9b774-lkk6r" (UID: "095be993-f7f9-11e9-8756-525400d144f2")
Oct 26 14:00:58 ctl01 kubelet[10393]: I1026 14:00:58.923967   10393 reconciler.go:252] operationExecutor.MountVolume started for volume "coredns-token-8cvdr" (UniqueName: "kubernetes.io/secret/09632cce-f7f9-11e9-8756-525400d144f2-coredns-token-8cvdr") pod "coredns-7f8f94c97b-fkm4q" (UID: "09632cce-f7f9-11e9-8756-525400d144f2")
Oct 26 14:00:58 ctl01 kubelet[10393]: I1026 14:00:58.923999   10393 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "netchecker-token-qcn6t" (UniqueName: "kubernetes.io/secret/09723ce9-f7f9-11e9-8756-525400d144f2-netchecker-token-qcn6t") pod "netchecker-server-7876fb46d4-r46ls" (UID: "09723ce9-f7f9-11e9-8756-525400d144f2")
Oct 26 14:00:58 ctl01 kubelet[10393]: I1026 14:00:58.924031   10393 reconciler.go:252] operationExecutor.MountVolume started for volume "config-volume" (UniqueName: "kubernetes.io/configmap/09632cce-f7f9-11e9-8756-525400d144f2-config-volume") pod "coredns-7f8f94c97b-fkm4q" (UID: "09632cce-f7f9-11e9-8756-525400d144f2")
Oct 26 14:00:58 ctl01 kubelet[10393]: I1026 14:00:58.924056   10393 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "etcd-certs" (UniqueName: "kubernetes.io/host-path/09723ce9-f7f9-11e9-8756-525400d144f2-etcd-certs") pod "netchecker-server-7876fb46d4-r46ls" (UID: "09723ce9-f7f9-11e9-8756-525400d144f2")
Oct 26 14:00:58 ctl01 kubelet[10393]: I1026 14:00:58.924188   10393 operation_generator.go:571] MountVolume.SetUp succeeded for volume "etcd-certs" (UniqueName: "kubernetes.io/host-path/095be993-f7f9-11e9-8756-525400d144f2-etcd-certs") pod "calico-kube-controllers-996f9b774-lkk6r" (UID: "095be993-f7f9-11e9-8756-525400d144f2")
Oct 26 14:00:58 ctl01 kubelet[10393]: I1026 14:00:58.926588   10393 operation_generator.go:571] MountVolume.SetUp succeeded for volume "config-volume" (UniqueName: "kubernetes.io/configmap/09632cce-f7f9-11e9-8756-525400d144f2-config-volume") pod "coredns-7f8f94c97b-fkm4q" (UID: "09632cce-f7f9-11e9-8756-525400d144f2")
Oct 26 14:00:58 ctl01 systemd[1]: Started Kubernetes transient mount for /var/lib/kubelet/pods/095be993-f7f9-11e9-8756-525400d144f2/volumes/kubernetes.io~secret/calico-kube-controllers-token-gxwp4.
Oct 26 14:00:58 ctl01 kubelet[10393]: I1026 14:00:58.937944   10393 operation_generator.go:571] MountVolume.SetUp succeeded for volume "calico-kube-controllers-token-gxwp4" (UniqueName: "kubernetes.io/secret/095be993-f7f9-11e9-8756-525400d144f2-calico-kube-controllers-token-gxwp4") pod "calico-kube-controllers-996f9b774-lkk6r" (UID: "095be993-f7f9-11e9-8756-525400d144f2")
Oct 26 14:00:58 ctl01 systemd[1]: Started Kubernetes transient mount for /var/lib/kubelet/pods/09632cce-f7f9-11e9-8756-525400d144f2/volumes/kubernetes.io~secret/coredns-token-8cvdr.
Oct 26 14:00:58 ctl01 kubelet[10393]: I1026 14:00:58.947369   10393 operation_generator.go:571] MountVolume.SetUp succeeded for volume "coredns-token-8cvdr" (UniqueName: "kubernetes.io/secret/09632cce-f7f9-11e9-8756-525400d144f2-coredns-token-8cvdr") pod "coredns-7f8f94c97b-fkm4q" (UID: "09632cce-f7f9-11e9-8756-525400d144f2")
Oct 26 14:00:59 ctl01 kubelet[10393]: I1026 14:00:59.024395   10393 reconciler.go:252] operationExecutor.MountVolume started for volume "netchecker-token-qcn6t" (UniqueName: "kubernetes.io/secret/09723ce9-f7f9-11e9-8756-525400d144f2-netchecker-token-qcn6t") pod "netchecker-server-7876fb46d4-r46ls" (UID: "09723ce9-f7f9-11e9-8756-525400d144f2")
Oct 26 14:00:59 ctl01 kubelet[10393]: I1026 14:00:59.024484   10393 reconciler.go:252] operationExecutor.MountVolume started for volume "etcd-certs" (UniqueName: "kubernetes.io/host-path/09723ce9-f7f9-11e9-8756-525400d144f2-etcd-certs") pod "netchecker-server-7876fb46d4-r46ls" (UID: "09723ce9-f7f9-11e9-8756-525400d144f2")
Oct 26 14:00:59 ctl01 kubelet[10393]: I1026 14:00:59.024580   10393 operation_generator.go:571] MountVolume.SetUp succeeded for volume "etcd-certs" (UniqueName: "kubernetes.io/host-path/09723ce9-f7f9-11e9-8756-525400d144f2-etcd-certs") pod "netchecker-server-7876fb46d4-r46ls" (UID: "09723ce9-f7f9-11e9-8756-525400d144f2")
Oct 26 14:00:59 ctl01 systemd[1]: Started Kubernetes transient mount for /var/lib/kubelet/pods/09723ce9-f7f9-11e9-8756-525400d144f2/volumes/kubernetes.io~secret/netchecker-token-qcn6t.
Oct 26 14:00:59 ctl01 kubelet[10393]: I1026 14:00:59.040818   10393 operation_generator.go:571] MountVolume.SetUp succeeded for volume "netchecker-token-qcn6t" (UniqueName: "kubernetes.io/secret/09723ce9-f7f9-11e9-8756-525400d144f2-netchecker-token-qcn6t") pod "netchecker-server-7876fb46d4-r46ls" (UID: "09723ce9-f7f9-11e9-8756-525400d144f2")
Oct 26 14:00:59 ctl01 kubelet[10393]: W1026 14:00:59.050527   10393 container.go:409] Failed to create summary reader for "/system.slice/run-r0fc0c28f783b49b3bff6a990386d4418.scope": none of the resources are being tracked.
Oct 26 14:00:59 ctl01 kubelet[10393]: W1026 14:00:59.050651   10393 raw.go:87] Error while processing event ("/sys/fs/cgroup/cpu,cpuacct/system.slice/run-rfeed7b6b9ff9435b8f9bff304e4f1d08.scope": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/cpu,cpuacct/system.slice/run-rfeed7b6b9ff9435b8f9bff304e4f1d08.scope: no such file or directory
Oct 26 14:00:59 ctl01 kubelet[10393]: W1026 14:00:59.050702   10393 raw.go:87] Error while processing event ("/sys/fs/cgroup/blkio/system.slice/run-rfeed7b6b9ff9435b8f9bff304e4f1d08.scope": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/blkio/system.slice/run-rfeed7b6b9ff9435b8f9bff304e4f1d08.scope: no such file or directory
Oct 26 14:00:59 ctl01 kubelet[10393]: W1026 14:00:59.050730   10393 raw.go:87] Error while processing event ("/sys/fs/cgroup/memory/system.slice/run-rfeed7b6b9ff9435b8f9bff304e4f1d08.scope": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/memory/system.slice/run-rfeed7b6b9ff9435b8f9bff304e4f1d08.scope: no such file or directory
Oct 26 14:00:59 ctl01 kubelet[10393]: W1026 14:00:59.050755   10393 raw.go:87] Error while processing event ("/sys/fs/cgroup/devices/system.slice/run-rfeed7b6b9ff9435b8f9bff304e4f1d08.scope": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/devices/system.slice/run-rfeed7b6b9ff9435b8f9bff304e4f1d08.scope: no such file or directory
Oct 26 14:00:59 ctl01 kubelet[10393]: W1026 14:00:59.050829   10393 raw.go:87] Error while processing event ("/sys/fs/cgroup/cpu,cpuacct/system.slice/run-r34c74aad0a464b4592558d9f89e34787.scope": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/cpu,cpuacct/system.slice/run-r34c74aad0a464b4592558d9f89e34787.scope: no such file or directory
Oct 26 14:00:59 ctl01 kubelet[10393]: W1026 14:00:59.050857   10393 raw.go:87] Error while processing event ("/sys/fs/cgroup/blkio/system.slice/run-r34c74aad0a464b4592558d9f89e34787.scope": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/blkio/system.slice/run-r34c74aad0a464b4592558d9f89e34787.scope: no such file or directory
Oct 26 14:00:59 ctl01 kubelet[10393]: W1026 14:00:59.050879   10393 raw.go:87] Error while processing event ("/sys/fs/cgroup/memory/system.slice/run-r34c74aad0a464b4592558d9f89e34787.scope": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/memory/system.slice/run-r34c74aad0a464b4592558d9f89e34787.scope: no such file or directory
Oct 26 14:00:59 ctl01 kubelet[10393]: W1026 14:00:59.050903   10393 raw.go:87] Error while processing event ("/sys/fs/cgroup/devices/system.slice/run-r34c74aad0a464b4592558d9f89e34787.scope": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/devices/system.slice/run-r34c74aad0a464b4592558d9f89e34787.scope: no such file or directory
Oct 26 14:00:59 ctl01 kubelet[10393]: I1026 14:00:59.061648   10393 kuberuntime_manager.go:397] No sandbox for pod "calico-kube-controllers-996f9b774-lkk6r_kube-system(095be993-f7f9-11e9-8756-525400d144f2)" can be found. Need to start a new one
Oct 26 14:00:59 ctl01 containerd[6508]: time="2019-10-26T14:00:59.062263922Z" level=info msg="RunPodsandbox for &PodSandboxMetadata{Name:calico-kube-controllers-996f9b774-lkk6r,Uid:095be993-f7f9-11e9-8756-525400d144f2,Namespace:kube-system,Attempt:0,}"
Oct 26 14:00:59 ctl01 containerd[6508]: time="2019-10-26T14:00:59.070667535Z" level=info msg="shim containerd-shim started" address="/containerd-shim/k8s.io/b8769b61f2d431995502c07ac6b9a136ce98b7528a601ea5a3eb31d06aaf42d0/shim.sock" debug=false pid=12170
Oct 26 14:00:59 ctl01 kubelet[10393]: I1026 14:00:59.106065   10393 kuberuntime_manager.go:397] No sandbox for pod "coredns-7f8f94c97b-fkm4q_kube-system(09632cce-f7f9-11e9-8756-525400d144f2)" can be found. Need to start a new one
Oct 26 14:00:59 ctl01 containerd[6508]: time="2019-10-26T14:00:59.106551380Z" level=info msg="RunPodsandbox for &PodSandboxMetadata{Name:coredns-7f8f94c97b-fkm4q,Uid:09632cce-f7f9-11e9-8756-525400d144f2,Namespace:kube-system,Attempt:0,}"
Oct 26 14:00:59 ctl01 kubelet[10393]: I1026 14:00:59.209603   10393 kuberuntime_manager.go:397] No sandbox for pod "netchecker-server-7876fb46d4-r46ls_netchecker(09723ce9-f7f9-11e9-8756-525400d144f2)" can be found. Need to start a new one
Oct 26 14:00:59 ctl01 containerd[6508]: time="2019-10-26T14:00:59.210637940Z" level=info msg="RunPodsandbox for &PodSandboxMetadata{Name:netchecker-server-7876fb46d4-r46ls,Uid:09723ce9-f7f9-11e9-8756-525400d144f2,Namespace:netchecker,Attempt:0,}"
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.396 [INFO][12220] calico.go 75: Extracted identifiers EndpointIDs=&utils.WEPIdentifiers{Namespace:"kube-system", WEPName:"", WorkloadEndpointIdentifiers:names.WorkloadEndpointIdentifiers{Node:"ctl01", Orchestrator:"k8s", Endpoint:"eth0", Workload:"", Pod:"coredns-7f8f94c97b-fkm4q", ContainerID:"21c403d649a680c2f23874b7600ac662e4cf0b12fb206be52c2b7d38694efc41"}}
Oct 26 14:00:59 ctl01 containerd[6508]: time="2019-10-26T14:00:59.457511019Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-996f9b774-lkk6r,Uid:095be993-f7f9-11e9-8756-525400d144f2,Namespace:kube-system,Attempt:0,} returns sandbox id "b8769b61f2d431995502c07ac6b9a136ce98b7528a601ea5a3eb31d06aaf42d0""
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.458 [INFO][12220] k8s.go 60: Extracted identifiers for CmdAddK8s ContainerID="21c403d649a680c2f23874b7600ac662e4cf0b12fb206be52c2b7d38694efc41" Namespace="kube-system" Pod="coredns-7f8f94c97b-fkm4q" WorkloadEndpoint="ctl01-k8s-coredns--7f8f94c97b--fkm4q-eth0"
Oct 26 14:00:59 ctl01 kubelet[10393]: I1026 14:00:59.536008   10393 kubelet.go:1953] SyncLoop (PLEG): "netchecker-agent-5mrgs_netchecker(08d4a4df-f7f9-11e9-8756-525400d144f2)", event: &pleg.PodLifecycleEvent{ID:"08d4a4df-f7f9-11e9-8756-525400d144f2", Type:"ContainerStarted", Data:"b875754565fd9f2dcdab462102a962ed536b19fab9397471ec6c682c28883867"}
Oct 26 14:00:59 ctl01 kubelet[10393]: I1026 14:00:59.537681   10393 kubelet.go:1953] SyncLoop (PLEG): "calico-kube-controllers-996f9b774-lkk6r_kube-system(095be993-f7f9-11e9-8756-525400d144f2)", event: &pleg.PodLifecycleEvent{ID:"095be993-f7f9-11e9-8756-525400d144f2", Type:"ContainerStarted", Data:"b8769b61f2d431995502c07ac6b9a136ce98b7528a601ea5a3eb31d06aaf42d0"}
Oct 26 14:00:59 ctl01 containerd[6508]: Calico CNI IPAM request count IPv4=1 IPv6=0
Oct 26 14:00:59 ctl01 containerd[6508]: Calico CNI IPAM handle=calico-k8s-network.21c403d649a680c2f23874b7600ac662e4cf0b12fb206be52c2b7d38694efc41
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.552 [INFO][12255] calico-ipam.go 186: Auto assigning IP ContainerID="21c403d649a680c2f23874b7600ac662e4cf0b12fb206be52c2b7d38694efc41" HandleID="calico-k8s-network.21c403d649a680c2f23874b7600ac662e4cf0b12fb206be52c2b7d38694efc41" Workload="ctl01-k8s-coredns--7f8f94c97b--fkm4q-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc4202281d0), Attrs:map[string]string(nil), Hostname:"ctl01", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}}
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.552 [INFO][12255] ipam.go 70: Auto-assign 1 ipv4, 0 ipv6 addrs for host 'ctl01'
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.553 [INFO][12255] ipam.go 254: Looking up existing affinities for host handle="calico-k8s-network.21c403d649a680c2f23874b7600ac662e4cf0b12fb206be52c2b7d38694efc41" host="ctl01"
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.554 [INFO][12255] ipam.go 275: Trying affinity for 192.168.237.0/26 handle="calico-k8s-network.21c403d649a680c2f23874b7600ac662e4cf0b12fb206be52c2b7d38694efc41" host="ctl01"
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.555 [INFO][12255] ipam.go 118: Attempting to load block cidr=192.168.237.0/26 host="ctl01"
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.557 [INFO][12255] ipam.go 195: Affinity is confirmed and block has been loaded cidr=192.168.237.0/26 host="ctl01"
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.557 [INFO][12255] ipam.go 677: Attempting to assign 1 addresses from block block=192.168.237.0/26 handle="calico-k8s-network.21c403d649a680c2f23874b7600ac662e4cf0b12fb206be52c2b7d38694efc41" host="ctl01"
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.558 [INFO][12255] ipam.go 1110: Creating new handle: calico-k8s-network.21c403d649a680c2f23874b7600ac662e4cf0b12fb206be52c2b7d38694efc41
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.560 [INFO][12255] ipam.go 700: Writing block in order to claim IPs block=192.168.237.0/26 handle="calico-k8s-network.21c403d649a680c2f23874b7600ac662e4cf0b12fb206be52c2b7d38694efc41" host="ctl01"
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.561 [INFO][12255] ipam.go 710: Successfully claimed IPs: [192.168.237.1] block=192.168.237.0/26 handle="calico-k8s-network.21c403d649a680c2f23874b7600ac662e4cf0b12fb206be52c2b7d38694efc41" host="ctl01"
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.561 [INFO][12255] ipam.go 307: Block '192.168.237.0/26' provided addresses: [192.168.237.1] handle="calico-k8s-network.21c403d649a680c2f23874b7600ac662e4cf0b12fb206be52c2b7d38694efc41" host="ctl01"
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.562 [INFO][12255] ipam.go 456: Auto-assigned 1 out of 1 IPv4s: [192.168.237.1] handle="calico-k8s-network.21c403d649a680c2f23874b7600ac662e4cf0b12fb206be52c2b7d38694efc41" host="ctl01"
Oct 26 14:00:59 ctl01 containerd[6508]: Calico CNI IPAM assigned addresses IPv4=[192.168.237.1] IPv6=[]
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.562 [INFO][12255] calico-ipam.go 214: IPAM Result ContainerID="21c403d649a680c2f23874b7600ac662e4cf0b12fb206be52c2b7d38694efc41" HandleID="calico-k8s-network.21c403d649a680c2f23874b7600ac662e4cf0b12fb206be52c2b7d38694efc41" Workload="ctl01-k8s-coredns--7f8f94c97b--fkm4q-eth0" result.IPs=[]*current.IPConfig{(*current.IPConfig)(0xc420694060)}
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.564 [INFO][12220] k8s.go 365: Populated endpoint ContainerID="21c403d649a680c2f23874b7600ac662e4cf0b12fb206be52c2b7d38694efc41" Namespace="kube-system" Pod="coredns-7f8f94c97b-fkm4q" WorkloadEndpoint="ctl01-k8s-coredns--7f8f94c97b--fkm4q-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"ctl01-k8s-coredns--7f8f94c97b--fkm4q-eth0", GenerateName:"", Namespace:"kube-system", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"ctl01", ContainerID:"", Pod:"coredns-7f8f94c97b-fkm4q", Endpoint:"eth0", IPNetworks:[]string{"192.168.237.1/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"calico-k8s-network"}, InterfaceName:"", MAC:"", Ports:[]v3.EndpointPort(nil)}}
Oct 26 14:00:59 ctl01 containerd[6508]: Calico CNI using IPs: [192.168.237.1/32]
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.564 [INFO][12220] network.go 75: Setting the host side veth name to calib68953dea40 ContainerID="21c403d649a680c2f23874b7600ac662e4cf0b12fb206be52c2b7d38694efc41" Namespace="kube-system" Pod="coredns-7f8f94c97b-fkm4q" WorkloadEndpoint="ctl01-k8s-coredns--7f8f94c97b--fkm4q-eth0"
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.566 [INFO][12220] network.go 380: Disabling IPv4 forwarding ContainerID="21c403d649a680c2f23874b7600ac662e4cf0b12fb206be52c2b7d38694efc41" Namespace="kube-system" Pod="coredns-7f8f94c97b-fkm4q" WorkloadEndpoint="ctl01-k8s-coredns--7f8f94c97b--fkm4q-eth0"
Oct 26 14:00:59 ctl01 kernel: [  257.252528] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
Oct 26 14:00:59 ctl01 kernel: [  257.252951] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.592 [INFO][12220] k8s.go 392: Added Mac, interface name, and active container ID to endpoint ContainerID="21c403d649a680c2f23874b7600ac662e4cf0b12fb206be52c2b7d38694efc41" Namespace="kube-system" Pod="coredns-7f8f94c97b-fkm4q" WorkloadEndpoint="ctl01-k8s-coredns--7f8f94c97b--fkm4q-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"ctl01-k8s-coredns--7f8f94c97b--fkm4q-eth0", GenerateName:"", Namespace:"kube-system", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"ctl01", ContainerID:"21c403d649a680c2f23874b7600ac662e4cf0b12fb206be52c2b7d38694efc41", Pod:"coredns-7f8f94c97b-fkm4q", Endpoint:"eth0", IPNetworks:[]string{"192.168.237.1/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"calico-k8s-network"}, InterfaceName:"calib68953dea40", MAC:"36:6f:a1:e0:28:25", Ports:[]v3.EndpointPort(nil)}}
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.599 [INFO][12220] k8s.go 424: Wrote updated endpoint to datastore ContainerID="21c403d649a680c2f23874b7600ac662e4cf0b12fb206be52c2b7d38694efc41" Namespace="kube-system" Pod="coredns-7f8f94c97b-fkm4q" WorkloadEndpoint="ctl01-k8s-coredns--7f8f94c97b--fkm4q-eth0"
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.623 [INFO][12285] calico.go 75: Extracted identifiers EndpointIDs=&utils.WEPIdentifiers{Namespace:"netchecker", WEPName:"", WorkloadEndpointIdentifiers:names.WorkloadEndpointIdentifiers{Node:"ctl01", Orchestrator:"k8s", Endpoint:"eth0", Workload:"", Pod:"netchecker-server-7876fb46d4-r46ls", ContainerID:"23994f34def3027d5b06e431f814722a915f285a5500217637a5f7a946ba6035"}}
Oct 26 14:00:59 ctl01 containerd[6508]: time="2019-10-26T14:00:59.630246434Z" level=info msg="shim containerd-shim started" address="/containerd-shim/k8s.io/21c403d649a680c2f23874b7600ac662e4cf0b12fb206be52c2b7d38694efc41/shim.sock" debug=false pid=12344
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.687 [INFO][12285] k8s.go 60: Extracted identifiers for CmdAddK8s ContainerID="23994f34def3027d5b06e431f814722a915f285a5500217637a5f7a946ba6035" Namespace="netchecker" Pod="netchecker-server-7876fb46d4-r46ls" WorkloadEndpoint="ctl01-k8s-netchecker--server--7876fb46d4--r46ls-eth0"
Oct 26 14:00:59 ctl01 containerd[6508]: Calico CNI IPAM request count IPv4=1 IPv6=0
Oct 26 14:00:59 ctl01 containerd[6508]: Calico CNI IPAM handle=calico-k8s-network.23994f34def3027d5b06e431f814722a915f285a5500217637a5f7a946ba6035
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.772 [INFO][12385] calico-ipam.go 186: Auto assigning IP ContainerID="23994f34def3027d5b06e431f814722a915f285a5500217637a5f7a946ba6035" HandleID="calico-k8s-network.23994f34def3027d5b06e431f814722a915f285a5500217637a5f7a946ba6035" Workload="ctl01-k8s-netchecker--server--7876fb46d4--r46ls-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc420081250), Attrs:map[string]string(nil), Hostname:"ctl01", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}}
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.772 [INFO][12385] ipam.go 70: Auto-assign 1 ipv4, 0 ipv6 addrs for host 'ctl01'
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.773 [INFO][12385] ipam.go 254: Looking up existing affinities for host handle="calico-k8s-network.23994f34def3027d5b06e431f814722a915f285a5500217637a5f7a946ba6035" host="ctl01"
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.775 [INFO][12385] ipam.go 275: Trying affinity for 192.168.237.0/26 handle="calico-k8s-network.23994f34def3027d5b06e431f814722a915f285a5500217637a5f7a946ba6035" host="ctl01"
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.776 [INFO][12385] ipam.go 118: Attempting to load block cidr=192.168.237.0/26 host="ctl01"
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.777 [INFO][12385] ipam.go 195: Affinity is confirmed and block has been loaded cidr=192.168.237.0/26 host="ctl01"
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.777 [INFO][12385] ipam.go 677: Attempting to assign 1 addresses from block block=192.168.237.0/26 handle="calico-k8s-network.23994f34def3027d5b06e431f814722a915f285a5500217637a5f7a946ba6035" host="ctl01"
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.778 [INFO][12385] ipam.go 1110: Creating new handle: calico-k8s-network.23994f34def3027d5b06e431f814722a915f285a5500217637a5f7a946ba6035
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.779 [INFO][12385] ipam.go 700: Writing block in order to claim IPs block=192.168.237.0/26 handle="calico-k8s-network.23994f34def3027d5b06e431f814722a915f285a5500217637a5f7a946ba6035" host="ctl01"
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.780 [INFO][12385] ipam.go 710: Successfully claimed IPs: [192.168.237.2] block=192.168.237.0/26 handle="calico-k8s-network.23994f34def3027d5b06e431f814722a915f285a5500217637a5f7a946ba6035" host="ctl01"
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.780 [INFO][12385] ipam.go 307: Block '192.168.237.0/26' provided addresses: [192.168.237.2] handle="calico-k8s-network.23994f34def3027d5b06e431f814722a915f285a5500217637a5f7a946ba6035" host="ctl01"
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.781 [INFO][12385] ipam.go 456: Auto-assigned 1 out of 1 IPv4s: [192.168.237.2] handle="calico-k8s-network.23994f34def3027d5b06e431f814722a915f285a5500217637a5f7a946ba6035" host="ctl01"
Oct 26 14:00:59 ctl01 containerd[6508]: Calico CNI IPAM assigned addresses IPv4=[192.168.237.2] IPv6=[]
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.781 [INFO][12385] calico-ipam.go 214: IPAM Result ContainerID="23994f34def3027d5b06e431f814722a915f285a5500217637a5f7a946ba6035" HandleID="calico-k8s-network.23994f34def3027d5b06e431f814722a915f285a5500217637a5f7a946ba6035" Workload="ctl01-k8s-netchecker--server--7876fb46d4--r46ls-eth0" result.IPs=[]*current.IPConfig{(*current.IPConfig)(0xc4201244e0)}
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.783 [INFO][12285] k8s.go 365: Populated endpoint ContainerID="23994f34def3027d5b06e431f814722a915f285a5500217637a5f7a946ba6035" Namespace="netchecker" Pod="netchecker-server-7876fb46d4-r46ls" WorkloadEndpoint="ctl01-k8s-netchecker--server--7876fb46d4--r46ls-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"ctl01-k8s-netchecker--server--7876fb46d4--r46ls-eth0", GenerateName:"", Namespace:"netchecker", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"ctl01", ContainerID:"", Pod:"netchecker-server-7876fb46d4-r46ls", Endpoint:"eth0", IPNetworks:[]string{"192.168.237.2/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"calico-k8s-network"}, InterfaceName:"", MAC:"", Ports:[]v3.EndpointPort(nil)}}
Oct 26 14:00:59 ctl01 containerd[6508]: Calico CNI using IPs: [192.168.237.2/32]
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.783 [INFO][12285] network.go 75: Setting the host side veth name to calie8cee6818cb ContainerID="23994f34def3027d5b06e431f814722a915f285a5500217637a5f7a946ba6035" Namespace="netchecker" Pod="netchecker-server-7876fb46d4-r46ls" WorkloadEndpoint="ctl01-k8s-netchecker--server--7876fb46d4--r46ls-eth0"
Oct 26 14:00:59 ctl01 kernel: [  257.470912] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
Oct 26 14:00:59 ctl01 kernel: [  257.471131] IPv6: ADDRCONF(NETDEV_UP): calie8cee6818cb: link is not ready
Oct 26 14:00:59 ctl01 kernel: [  257.471138] IPv6: ADDRCONF(NETDEV_CHANGE): calie8cee6818cb: link becomes ready
Oct 26 14:00:59 ctl01 kernel: [  257.471170] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.786 [INFO][12285] network.go 380: Disabling IPv4 forwarding ContainerID="23994f34def3027d5b06e431f814722a915f285a5500217637a5f7a946ba6035" Namespace="netchecker" Pod="netchecker-server-7876fb46d4-r46ls" WorkloadEndpoint="ctl01-k8s-netchecker--server--7876fb46d4--r46ls-eth0"
Oct 26 14:00:59 ctl01 containerd[6508]: time="2019-10-26T14:00:59.806054946Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-7f8f94c97b-fkm4q,Uid:09632cce-f7f9-11e9-8756-525400d144f2,Namespace:kube-system,Attempt:0,} returns sandbox id "21c403d649a680c2f23874b7600ac662e4cf0b12fb206be52c2b7d38694efc41""
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.811 [INFO][12285] k8s.go 392: Added Mac, interface name, and active container ID to endpoint ContainerID="23994f34def3027d5b06e431f814722a915f285a5500217637a5f7a946ba6035" Namespace="netchecker" Pod="netchecker-server-7876fb46d4-r46ls" WorkloadEndpoint="ctl01-k8s-netchecker--server--7876fb46d4--r46ls-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"ctl01-k8s-netchecker--server--7876fb46d4--r46ls-eth0", GenerateName:"", Namespace:"netchecker", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"ctl01", ContainerID:"23994f34def3027d5b06e431f814722a915f285a5500217637a5f7a946ba6035", Pod:"netchecker-server-7876fb46d4-r46ls", Endpoint:"eth0", IPNetworks:[]string{"192.168.237.2/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"calico-k8s-network"}, InterfaceName:"calie8cee6818cb", MAC:"d6:cf:6a:81:a5:d9", Ports:[]v3.EndpointPort(nil)}}
Oct 26 14:00:59 ctl01 containerd[6508]: 2019-10-26 14:00:59.826 [INFO][12285] k8s.go 424: Wrote updated endpoint to datastore ContainerID="23994f34def3027d5b06e431f814722a915f285a5500217637a5f7a946ba6035" Namespace="netchecker" Pod="netchecker-server-7876fb46d4-r46ls" WorkloadEndpoint="ctl01-k8s-netchecker--server--7876fb46d4--r46ls-eth0"
Oct 26 14:00:59 ctl01 containerd[6508]: time="2019-10-26T14:00:59.845169411Z" level=info msg="shim containerd-shim started" address="/containerd-shim/k8s.io/23994f34def3027d5b06e431f814722a915f285a5500217637a5f7a946ba6035/shim.sock" debug=false pid=12470
Oct 26 14:01:00 ctl01 containerd[6508]: time="2019-10-26T14:01:00.031590661Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:netchecker-server-7876fb46d4-r46ls,Uid:09723ce9-f7f9-11e9-8756-525400d144f2,Namespace:netchecker,Attempt:0,} returns sandbox id "23994f34def3027d5b06e431f814722a915f285a5500217637a5f7a946ba6035""
Oct 26 14:01:00 ctl01 kubelet[10393]: I1026 14:01:00.541926   10393 kubelet.go:1953] SyncLoop (PLEG): "coredns-7f8f94c97b-fkm4q_kube-system(09632cce-f7f9-11e9-8756-525400d144f2)", event: &pleg.PodLifecycleEvent{ID:"09632cce-f7f9-11e9-8756-525400d144f2", Type:"ContainerStarted", Data:"21c403d649a680c2f23874b7600ac662e4cf0b12fb206be52c2b7d38694efc41"}
Oct 26 14:01:00 ctl01 kubelet[10393]: I1026 14:01:00.545488   10393 kubelet.go:1953] SyncLoop (PLEG): "netchecker-server-7876fb46d4-r46ls_netchecker(09723ce9-f7f9-11e9-8756-525400d144f2)", event: &pleg.PodLifecycleEvent{ID:"09723ce9-f7f9-11e9-8756-525400d144f2", Type:"ContainerStarted", Data:"23994f34def3027d5b06e431f814722a915f285a5500217637a5f7a946ba6035"}
Oct 26 14:01:00 ctl01 containerd[6508]: time="2019-10-26T14:01:00.773618783Z" level=info msg="ImageCreate event &ImageCreate{Name:docker.io/mirantis/k8s-netchecker-agent:stable,Labels:map[string]string{},}"
Oct 26 14:01:00 ctl01 containerd[6508]: time="2019-10-26T14:01:00.781158125Z" level=info msg="ImageCreate event &ImageCreate{Name:sha256:d16c8d7f4d5bdaecda098c54c655027871da52f292683759f5a32c8ecd43b3f0,Labels:map[string]string{io.cri-containerd.image: managed,},}"
Oct 26 14:01:00 ctl01 containerd[6508]: time="2019-10-26T14:01:00.782150965Z" level=info msg="ImageUpdate event &ImageUpdate{Name:docker.io/mirantis/k8s-netchecker-agent:stable,Labels:map[string]string{io.cri-containerd.image: managed,},}"
Oct 26 14:01:01 ctl01 containerd[6508]: time="2019-10-26T14:01:01.031839916Z" level=info msg="ImageUpdate event &ImageUpdate{Name:sha256:d16c8d7f4d5bdaecda098c54c655027871da52f292683759f5a32c8ecd43b3f0,Labels:map[string]string{io.cri-containerd.image: managed,},}"
Oct 26 14:01:01 ctl01 containerd[6508]: time="2019-10-26T14:01:01.034930650Z" level=info msg="ImageUpdate event &ImageUpdate{Name:docker.io/mirantis/k8s-netchecker-agent:stable,Labels:map[string]string{io.cri-containerd.image: managed,},}"
Oct 26 14:01:01 ctl01 containerd[6508]: time="2019-10-26T14:01:01.037762348Z" level=info msg="ImageCreate event &ImageCreate{Name:docker.io/mirantis/k8s-netchecker-agent@sha256:4ac49ebef7eaeaa5a8a19f56faa73740bf4861979aa067d0125867a72846720a,Labels:map[string]string{io.cri-containerd.image: managed,},}"
Oct 26 14:01:01 ctl01 containerd[6508]: time="2019-10-26T14:01:01.038795879Z" level=info msg="PullImage "mirantis/k8s-netchecker-agent:stable" returns image reference "sha256:d16c8d7f4d5bdaecda098c54c655027871da52f292683759f5a32c8ecd43b3f0""
Oct 26 14:01:01 ctl01 containerd[6508]: time="2019-10-26T14:01:01.041146113Z" level=info msg="PullImage "docker-prod-local.artifactory.mirantis.com/mirantis/projectcalico/calico/kube-controllers:v3.3.2""
Oct 26 14:01:01 ctl01 containerd[6508]: time="2019-10-26T14:01:01.043772985Z" level=info msg="CreateContainer within sandbox "b875754565fd9f2dcdab462102a962ed536b19fab9397471ec6c682c28883867" for container &ContainerMetadata{Name:netchecker-agent,Attempt:0,}"
Oct 26 14:01:01 ctl01 containerd[6508]: time="2019-10-26T14:01:01.120603251Z" level=info msg="CreateContainer within sandbox "b875754565fd9f2dcdab462102a962ed536b19fab9397471ec6c682c28883867" for &ContainerMetadata{Name:netchecker-agent,Attempt:0,} returns container id "f41f637dae783cb289aae99f5e165ddba3bb77a43c0e08c5f99f5956f0ae2511""
Oct 26 14:01:01 ctl01 containerd[6508]: time="2019-10-26T14:01:01.121607415Z" level=info msg="StartContainer for "f41f637dae783cb289aae99f5e165ddba3bb77a43c0e08c5f99f5956f0ae2511""
Oct 26 14:01:01 ctl01 containerd[6508]: time="2019-10-26T14:01:01.122748861Z" level=info msg="shim containerd-shim started" address="/containerd-shim/k8s.io/f41f637dae783cb289aae99f5e165ddba3bb77a43c0e08c5f99f5956f0ae2511/shim.sock" debug=false pid=12571
Oct 26 14:01:01 ctl01 containerd[6508]: time="2019-10-26T14:01:01.319158013Z" level=info msg="StartContainer for "f41f637dae783cb289aae99f5e165ddba3bb77a43c0e08c5f99f5956f0ae2511" returns successfully"
Oct 26 14:01:01 ctl01 kubelet[10393]: I1026 14:01:01.551401   10393 kubelet.go:1953] SyncLoop (PLEG): "netchecker-agent-5mrgs_netchecker(08d4a4df-f7f9-11e9-8756-525400d144f2)", event: &pleg.PodLifecycleEvent{ID:"08d4a4df-f7f9-11e9-8756-525400d144f2", Type:"ContainerStarted", Data:"f41f637dae783cb289aae99f5e165ddba3bb77a43c0e08c5f99f5956f0ae2511"}
Oct 26 14:01:02 ctl01 containerd[6508]: time="2019-10-26T14:01:02.696564556Z" level=info msg="ImageCreate event &ImageCreate{Name:docker-prod-local.artifactory.mirantis.com/mirantis/projectcalico/calico/kube-controllers:v3.3.2,Labels:map[string]string{},}"
Oct 26 14:01:02 ctl01 containerd[6508]: time="2019-10-26T14:01:02.706657482Z" level=info msg="ImageCreate event &ImageCreate{Name:sha256:22c16a9aecce95ea9631683dd66ee0329e399e07bb86950239d4a6db1c6d201b,Labels:map[string]string{io.cri-containerd.image: managed,},}"
Oct 26 14:01:02 ctl01 containerd[6508]: time="2019-10-26T14:01:02.707627236Z" level=info msg="ImageUpdate event &ImageUpdate{Name:docker-prod-local.artifactory.mirantis.com/mirantis/projectcalico/calico/kube-controllers:v3.3.2,Labels:map[string]string{io.cri-containerd.image: managed,},}"
Oct 26 14:01:03 ctl01 containerd[6508]: time="2019-10-26T14:01:03.990643751Z" level=info msg="ImageUpdate event &ImageUpdate{Name:sha256:22c16a9aecce95ea9631683dd66ee0329e399e07bb86950239d4a6db1c6d201b,Labels:map[string]string{io.cri-containerd.image: managed,},}"
Oct 26 14:01:03 ctl01 containerd[6508]: time="2019-10-26T14:01:03.993091072Z" level=info msg="ImageUpdate event &ImageUpdate{Name:docker-prod-local.artifactory.mirantis.com/mirantis/projectcalico/calico/kube-controllers:v3.3.2,Labels:map[string]string{io.cri-containerd.image: managed,},}"
Oct 26 14:01:03 ctl01 containerd[6508]: time="2019-10-26T14:01:03.995053598Z" level=info msg="ImageCreate event &ImageCreate{Name:docker-prod-local.artifactory.mirantis.com/mirantis/projectcalico/calico/kube-controllers@sha256:59296255e44bcb3ab00dae468a795dd669c41d6d1f16b8b5a3f2a1271e88e810,Labels:map[string]string{io.cri-containerd.image: managed,},}"
Oct 26 14:01:03 ctl01 containerd[6508]: time="2019-10-26T14:01:03.995636557Z" level=info msg="PullImage "docker-prod-local.artifactory.mirantis.com/mirantis/projectcalico/calico/kube-controllers:v3.3.2" returns image reference "sha256:22c16a9aecce95ea9631683dd66ee0329e399e07bb86950239d4a6db1c6d201b""
Oct 26 14:01:03 ctl01 containerd[6508]: time="2019-10-26T14:01:03.996484807Z" level=info msg="PullImage "docker-prod-local.artifactory.mirantis.com/mirantis/coredns/coredns:v1.4.0-96""
Oct 26 14:01:03 ctl01 containerd[6508]: time="2019-10-26T14:01:03.997852269Z" level=info msg="CreateContainer within sandbox "b8769b61f2d431995502c07ac6b9a136ce98b7528a601ea5a3eb31d06aaf42d0" for container &ContainerMetadata{Name:calico-kube-controllers,Attempt:0,}"
Oct 26 14:01:04 ctl01 containerd[6508]: time="2019-10-26T14:01:04.101616987Z" level=info msg="CreateContainer within sandbox "b8769b61f2d431995502c07ac6b9a136ce98b7528a601ea5a3eb31d06aaf42d0" for &ContainerMetadata{Name:calico-kube-controllers,Attempt:0,} returns container id "cc51dd1565d8cb0ce6a8104facbeeb758af03f6866e4017949f5b569eb8d9b11""
Oct 26 14:01:04 ctl01 containerd[6508]: time="2019-10-26T14:01:04.102412051Z" level=info msg="StartContainer for "cc51dd1565d8cb0ce6a8104facbeeb758af03f6866e4017949f5b569eb8d9b11""
Oct 26 14:01:04 ctl01 containerd[6508]: time="2019-10-26T14:01:04.106545091Z" level=info msg="shim containerd-shim started" address="/containerd-shim/k8s.io/cc51dd1565d8cb0ce6a8104facbeeb758af03f6866e4017949f5b569eb8d9b11/shim.sock" debug=false pid=12656
Oct 26 14:01:04 ctl01 containerd[6508]: time="2019-10-26T14:01:04.531377691Z" level=info msg="StartContainer for "cc51dd1565d8cb0ce6a8104facbeeb758af03f6866e4017949f5b569eb8d9b11" returns successfully"
Oct 26 14:01:04 ctl01 kubelet[10393]: I1026 14:01:04.557300   10393 kubelet.go:1953] SyncLoop (PLEG): "calico-kube-controllers-996f9b774-lkk6r_kube-system(095be993-f7f9-11e9-8756-525400d144f2)", event: &pleg.PodLifecycleEvent{ID:"095be993-f7f9-11e9-8756-525400d144f2", Type:"ContainerStarted", Data:"cc51dd1565d8cb0ce6a8104facbeeb758af03f6866e4017949f5b569eb8d9b11"}
Oct 26 14:01:05 ctl01 containerd[6508]: time="2019-10-26T14:01:05.648200135Z" level=info msg="ImageCreate event &ImageCreate{Name:docker-prod-local.artifactory.mirantis.com/mirantis/coredns/coredns:v1.4.0-96,Labels:map[string]string{},}"
Oct 26 14:01:05 ctl01 containerd[6508]: time="2019-10-26T14:01:05.657116125Z" level=info msg="ImageCreate event &ImageCreate{Name:sha256:31b943183ba2eec561c44c1fdda2a39ab03c0cb72f90a24c1bea4adf0a44bfbc,Labels:map[string]string{io.cri-containerd.image: managed,},}"
Oct 26 14:01:05 ctl01 containerd[6508]: time="2019-10-26T14:01:05.658062391Z" level=info msg="ImageUpdate event &ImageUpdate{Name:docker-prod-local.artifactory.mirantis.com/mirantis/coredns/coredns:v1.4.0-96,Labels:map[string]string{io.cri-containerd.image: managed,},}"
Oct 26 14:01:05 ctl01 kube-proxy[8230]: I1026 14:01:05.822942    8230 proxier.go:659] Stale udp service kube-system/coredns:dns -> 10.254.0.10
Oct 26 14:01:05 ctl01 kernel: [  263.546049] ctnetlink v0.93: registering with nfnetlink.
Oct 26 14:01:06 ctl01 kubelet[10393]: I1026 14:01:06.144495   10393 setters.go:72] Using node IP: "172.16.10.36"
Oct 26 14:01:06 ctl01 containerd[6508]: time="2019-10-26T14:01:06.671536223Z" level=info msg="ImageUpdate event &ImageUpdate{Name:sha256:31b943183ba2eec561c44c1fdda2a39ab03c0cb72f90a24c1bea4adf0a44bfbc,Labels:map[string]string{io.cri-containerd.image: managed,},}"
Oct 26 14:01:06 ctl01 containerd[6508]: time="2019-10-26T14:01:06.673304023Z" level=info msg="ImageUpdate event &ImageUpdate{Name:docker-prod-local.artifactory.mirantis.com/mirantis/coredns/coredns:v1.4.0-96,Labels:map[string]string{io.cri-containerd.image: managed,},}"
Oct 26 14:01:06 ctl01 containerd[6508]: time="2019-10-26T14:01:06.674629321Z" level=info msg="ImageCreate event &ImageCreate{Name:docker-prod-local.artifactory.mirantis.com/mirantis/coredns/coredns@sha256:30f28dcd8a8c9a97c206bba187bd9b1e8fbc1cf52ce38f7e937c85a191709376,Labels:map[string]string{io.cri-containerd.image: managed,},}"
Oct 26 14:01:06 ctl01 containerd[6508]: time="2019-10-26T14:01:06.674985619Z" level=info msg="PullImage "docker-prod-local.artifactory.mirantis.com/mirantis/coredns/coredns:v1.4.0-96" returns image reference "sha256:31b943183ba2eec561c44c1fdda2a39ab03c0cb72f90a24c1bea4adf0a44bfbc""
Oct 26 14:01:06 ctl01 containerd[6508]: time="2019-10-26T14:01:06.675743657Z" level=info msg="PullImage "mirantis/k8s-netchecker-server:stable""
Oct 26 14:01:06 ctl01 containerd[6508]: time="2019-10-26T14:01:06.677131434Z" level=info msg="CreateContainer within sandbox "21c403d649a680c2f23874b7600ac662e4cf0b12fb206be52c2b7d38694efc41" for container &ContainerMetadata{Name:coredns,Attempt:0,}"
Oct 26 14:01:06 ctl01 containerd[6508]: time="2019-10-26T14:01:06.781343775Z" level=info msg="CreateContainer within sandbox "21c403d649a680c2f23874b7600ac662e4cf0b12fb206be52c2b7d38694efc41" for &ContainerMetadata{Name:coredns,Attempt:0,} returns container id "cee417a35c9d8f21b0e19fefd3cf573249c8e6768ee0a0607a00e7c387602a8d""
Oct 26 14:01:06 ctl01 containerd[6508]: time="2019-10-26T14:01:06.782536754Z" level=info msg="StartContainer for "cee417a35c9d8f21b0e19fefd3cf573249c8e6768ee0a0607a00e7c387602a8d""
Oct 26 14:01:06 ctl01 containerd[6508]: time="2019-10-26T14:01:06.783687476Z" level=info msg="shim containerd-shim started" address="/containerd-shim/k8s.io/cee417a35c9d8f21b0e19fefd3cf573249c8e6768ee0a0607a00e7c387602a8d/shim.sock" debug=false pid=12758
Oct 26 14:01:06 ctl01 containerd[6508]: time="2019-10-26T14:01:06.963401748Z" level=info msg="StartContainer for "cee417a35c9d8f21b0e19fefd3cf573249c8e6768ee0a0607a00e7c387602a8d" returns successfully"
Oct 26 14:01:07 ctl01 kubelet[10393]: I1026 14:01:07.564664   10393 kubelet.go:1953] SyncLoop (PLEG): "coredns-7f8f94c97b-fkm4q_kube-system(09632cce-f7f9-11e9-8756-525400d144f2)", event: &pleg.PodLifecycleEvent{ID:"09632cce-f7f9-11e9-8756-525400d144f2", Type:"ContainerStarted", Data:"cee417a35c9d8f21b0e19fefd3cf573249c8e6768ee0a0607a00e7c387602a8d"}
Oct 26 14:01:08 ctl01 containerd[6508]: time="2019-10-26T14:01:08.005664864Z" level=info msg="ImageCreate event &ImageCreate{Name:docker.io/mirantis/k8s-netchecker-server:stable,Labels:map[string]string{},}"
Oct 26 14:01:08 ctl01 containerd[6508]: time="2019-10-26T14:01:08.013142038Z" level=info msg="ImageCreate event &ImageCreate{Name:sha256:22d27d4fe076561ad51f5e7a0228ab7b1035773a492c95fd4a246b40ba4f9b58,Labels:map[string]string{io.cri-containerd.image: managed,},}"
Oct 26 14:01:08 ctl01 containerd[6508]: time="2019-10-26T14:01:08.014202002Z" level=info msg="ImageUpdate event &ImageUpdate{Name:docker.io/mirantis/k8s-netchecker-server:stable,Labels:map[string]string{io.cri-containerd.image: managed,},}"
Oct 26 14:01:11 ctl01 containerd[6508]: time="2019-10-26T14:01:11.249903647Z" level=info msg="ImageUpdate event &ImageUpdate{Name:sha256:22d27d4fe076561ad51f5e7a0228ab7b1035773a492c95fd4a246b40ba4f9b58,Labels:map[string]string{io.cri-containerd.image: managed,},}"
Oct 26 14:01:11 ctl01 containerd[6508]: time="2019-10-26T14:01:11.252114838Z" level=info msg="ImageUpdate event &ImageUpdate{Name:docker.io/mirantis/k8s-netchecker-server:stable,Labels:map[string]string{io.cri-containerd.image: managed,},}"
Oct 26 14:01:11 ctl01 containerd[6508]: time="2019-10-26T14:01:11.254454206Z" level=info msg="ImageCreate event &ImageCreate{Name:docker.io/mirantis/k8s-netchecker-server@sha256:87064b3cb72a8fb419eb714ae892ebd5929649e84499c08111e46e466acc8407,Labels:map[string]string{io.cri-containerd.image: managed,},}"
Oct 26 14:01:11 ctl01 containerd[6508]: time="2019-10-26T14:01:11.255072948Z" level=info msg="PullImage "mirantis/k8s-netchecker-server:stable" returns image reference "sha256:22d27d4fe076561ad51f5e7a0228ab7b1035773a492c95fd4a246b40ba4f9b58""
Oct 26 14:01:11 ctl01 containerd[6508]: time="2019-10-26T14:01:11.258789494Z" level=info msg="CreateContainer within sandbox "23994f34def3027d5b06e431f814722a915f285a5500217637a5f7a946ba6035" for container &ContainerMetadata{Name:netchecker-server,Attempt:0,}"
Oct 26 14:01:11 ctl01 containerd[6508]: time="2019-10-26T14:01:11.324631456Z" level=info msg="CreateContainer within sandbox "23994f34def3027d5b06e431f814722a915f285a5500217637a5f7a946ba6035" for &ContainerMetadata{Name:netchecker-server,Attempt:0,} returns container id "257aaa99e5c1ca1b87cc754453a0a4d867d748926e9159084f74e24e234e3f2c""
Oct 26 14:01:11 ctl01 containerd[6508]: time="2019-10-26T14:01:11.325396484Z" level=info msg="StartContainer for "257aaa99e5c1ca1b87cc754453a0a4d867d748926e9159084f74e24e234e3f2c""
Oct 26 14:01:11 ctl01 containerd[6508]: time="2019-10-26T14:01:11.326198106Z" level=info msg="shim containerd-shim started" address="/containerd-shim/k8s.io/257aaa99e5c1ca1b87cc754453a0a4d867d748926e9159084f74e24e234e3f2c/shim.sock" debug=false pid=12880
Oct 26 14:01:11 ctl01 containerd[6508]: time="2019-10-26T14:01:11.503486610Z" level=info msg="StartContainer for "257aaa99e5c1ca1b87cc754453a0a4d867d748926e9159084f74e24e234e3f2c" returns successfully"
Oct 26 14:01:11 ctl01 kubelet[10393]: I1026 14:01:11.573134   10393 kubelet.go:1953] SyncLoop (PLEG): "netchecker-server-7876fb46d4-r46ls_netchecker(09723ce9-f7f9-11e9-8756-525400d144f2)", event: &pleg.PodLifecycleEvent{ID:"09723ce9-f7f9-11e9-8756-525400d144f2", Type:"ContainerStarted", Data:"257aaa99e5c1ca1b87cc754453a0a4d867d748926e9159084f74e24e234e3f2c"}
Oct 26 14:01:16 ctl01 kubelet[10393]: I1026 14:01:16.154080   10393 setters.go:72] Using node IP: "172.16.10.36"
Oct 26 14:01:26 ctl01 kubelet[10393]: I1026 14:01:26.165181   10393 setters.go:72] Using node IP: "172.16.10.36"
Oct 26 14:01:36 ctl01 kubelet[10393]: I1026 14:01:36.178460   10393 setters.go:72] Using node IP: "172.16.10.36"
Oct 26 14:01:46 ctl01 kubelet[10393]: I1026 14:01:46.193989   10393 setters.go:72] Using node IP: "172.16.10.36"
Oct 26 14:01:56 ctl01 kubelet[10393]: I1026 14:01:56.204601   10393 setters.go:72] Using node IP: "172.16.10.36"
Oct 26 14:02:06 ctl01 kubelet[10393]: I1026 14:02:06.217606   10393 setters.go:72] Using node IP: "172.16.10.36"
Oct 26 14:02:16 ctl01 kubelet[10393]: I1026 14:02:16.228785   10393 setters.go:72] Using node IP: "172.16.10.36"
Oct 26 14:02:26 ctl01 kubelet[10393]: I1026 14:02:26.241985   10393 setters.go:72] Using node IP: "172.16.10.36"
Oct 26 14:02:36 ctl01 kubelet[10393]: I1026 14:02:36.253826   10393 setters.go:72] Using node IP: "172.16.10.36"
Oct 26 14:02:46 ctl01 kubelet[10393]: I1026 14:02:46.266114   10393 setters.go:72] Using node IP: "172.16.10.36"
Oct 26 14:02:56 ctl01 kubelet[10393]: I1026 14:02:56.275186   10393 setters.go:72] Using node IP: "172.16.10.36"
Oct 26 14:03:06 ctl01 kubelet[10393]: I1026 14:03:06.287327   10393 setters.go:72] Using node IP: "172.16.10.36"
Oct 26 14:03:16 ctl01 kubelet[10393]: I1026 14:03:16.298364   10393 setters.go:72] Using node IP: "172.16.10.36"
Oct 26 14:03:26 ctl01 kubelet[10393]: I1026 14:03:26.308482   10393 setters.go:72] Using node IP: "172.16.10.36"
Oct 26 14:03:34 ctl01 kube-apiserver[10762]: I1026 14:03:34.037698   10762 node_authorizer.go:176] NODE DENY: ctl01.mcp-k8s-calico-noha.local &authorizer.AttributesRecord{User:(*user.DefaultInfo)(0xc00a7a2500), Verb:"create", Namespace:"default", APIGroup:"", APIVersion:"v1", Resource:"configmaps", Subresource:"", Name:"", ResourceRequest:true, Path:"/api/v1/namespaces/default/configmaps"}
Oct 26 14:03:34 ctl01 kube-apiserver[10762]: I1026 14:03:34.125491   10762 node_authorizer.go:176] NODE DENY: ctl01.mcp-k8s-calico-noha.local &authorizer.AttributesRecord{User:(*user.DefaultInfo)(0xc008c1fec0), Verb:"delete", Namespace:"default", APIGroup:"", APIVersion:"v1", Resource:"configmaps", Subresource:"", Name:"k8s-b68f4919-key", ResourceRequest:true, Path:"/api/v1/namespaces/default/configmaps/k8s-b68f4919-key"}
Oct 26 14:03:34 ctl01 kube-apiserver[10762]: I1026 14:03:34.275928   10762 node_authorizer.go:176] NODE DENY: ctl01.mcp-k8s-calico-noha.local &authorizer.AttributesRecord{User:(*user.DefaultInfo)(0xc00ad7db40), Verb:"delete", Namespace:"default", APIGroup:"", APIVersion:"v1", Resource:"configmaps", Subresource:"", Name:"k8s-b68f4919-key", ResourceRequest:true, Path:"/api/v1/namespaces/default/configmaps/k8s-b68f4919-key"}
Oct 26 14:03:36 ctl01 kubelet[10393]: I1026 14:03:36.320576   10393 setters.go:72] Using node IP: "172.16.10.36"
Oct 26 14:03:46 ctl01 kubelet[10393]: I1026 14:03:46.334226   10393 setters.go:72] Using node IP: "172.16.10.36"
Oct 26 14:03:47 ctl01 salt-minion[4526]: [INFO    ] User sudo_ubuntu Executing command cp.push_dir with jid 20191026140347233165
Oct 26 14:03:47 ctl01 salt-minion[4526]: [INFO    ] Starting a new job with PID 13670
