==== start of pod log for container csi-mockplugin-attacher-0/csi-attacher ==== I0413 13:04:53.026552 1 main.go:91] Version: v2.2.0-0-g97411fa7 I0413 13:04:53.029027 1 connection.go:153] Connecting to unix:///csi/csi.sock I0413 13:04:55.210852 1 common.go:111] Probing CSI driver for readiness I0413 13:04:55.210893 1 connection.go:182] GRPC call: /csi.v1.Identity/Probe I0413 13:04:55.210902 1 connection.go:183] GRPC request: {} I0413 13:04:55.214405 1 connection.go:185] GRPC response: {"ready":{"value":true}} I0413 13:04:55.219665 1 connection.go:186] GRPC error: I0413 13:04:55.219693 1 connection.go:182] GRPC call: /csi.v1.Identity/GetPluginInfo I0413 13:04:55.219707 1 connection.go:183] GRPC request: {} I0413 13:04:55.221742 1 connection.go:185] GRPC response: {"manifest":{"url":"https://github.com/kubernetes-csi/csi-test/mock"},"name":"csi-mock-csi-mock-volumes-8039","vendor_version":"0.3.0"} I0413 13:04:55.222345 1 connection.go:186] GRPC error: I0413 13:04:55.222357 1 main.go:136] CSI driver name: "csi-mock-csi-mock-volumes-8039" W0413 13:04:55.222372 1 metrics.go:142] metrics endpoint will not be started because `metrics-address` was not specified. I0413 13:04:55.222380 1 connection.go:182] GRPC call: /csi.v1.Identity/GetPluginCapabilities I0413 13:04:55.222384 1 connection.go:183] GRPC request: {} I0413 13:04:55.224012 1 connection.go:185] GRPC response: {"capabilities":[{"Type":{"Service":{"type":1}}},{"Type":{"VolumeExpansion":{"type":1}}}]} I0413 13:04:55.226461 1 connection.go:186] GRPC error: I0413 13:04:55.226479 1 connection.go:182] GRPC call: /csi.v1.Controller/ControllerGetCapabilities I0413 13:04:55.226490 1 connection.go:183] GRPC request: {} I0413 13:04:55.228503 1 connection.go:185] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":3}}},{"Type":{"Rpc":{"type":10}}},{"Type":{"Rpc":{"type":4}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":8}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":12}}},{"Type":{"Rpc":{"type":11}}},{"Type":{"Rpc":{"type":2}}},{"Type":{"Rpc":{"type":9}}}]} I0413 13:04:55.235865 1 connection.go:186] GRPC error: I0413 13:04:55.235947 1 main.go:162] CSI driver supports ControllerPublishUnpublish, using real CSI handler I0413 13:04:55.235962 1 connection.go:182] GRPC call: /csi.v1.Controller/ControllerGetCapabilities I0413 13:04:55.235971 1 connection.go:183] GRPC request: {} I0413 13:04:55.237100 1 connection.go:185] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":3}}},{"Type":{"Rpc":{"type":10}}},{"Type":{"Rpc":{"type":4}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":8}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":12}}},{"Type":{"Rpc":{"type":11}}},{"Type":{"Rpc":{"type":2}}},{"Type":{"Rpc":{"type":9}}}]} I0413 13:04:55.246047 1 connection.go:186] GRPC error: I0413 13:04:55.246067 1 main.go:175] CSI driver supports list volumes published nodes. Using capability to reconcile volume attachment objects with actual backend state I0413 13:04:55.246227 1 controller.go:121] Starting CSI attacher I0413 13:04:55.246526 1 reflector.go:153] Starting reflector *v1beta1.VolumeAttachment (10m0s) from k8s.io/client-go/informers/factory.go:135 I0413 13:04:55.246554 1 reflector.go:188] Listing and watching *v1beta1.VolumeAttachment from k8s.io/client-go/informers/factory.go:135 I0413 13:04:55.246589 1 reflector.go:153] Starting reflector *v1beta1.CSINode (10m0s) from k8s.io/client-go/informers/factory.go:135 I0413 13:04:55.246589 1 reflector.go:153] Starting reflector *v1.PersistentVolume (10m0s) from k8s.io/client-go/informers/factory.go:135 I0413 13:04:55.246622 1 reflector.go:188] Listing and watching *v1.PersistentVolume from k8s.io/client-go/informers/factory.go:135 I0413 13:04:55.246611 1 reflector.go:188] Listing and watching *v1beta1.CSINode from k8s.io/client-go/informers/factory.go:135 I0413 13:04:55.346437 1 shared_informer.go:227] caches populated I0413 13:04:55.346543 1 csi_handler.go:116] Reconciling VolumeAttachments with driver backend state I0413 13:04:55.346589 1 connection.go:182] GRPC call: /csi.v1.Controller/ListVolumes I0413 13:04:55.346589 1 controller.go:198] Started VA processing "csi-b7952033dfdb180b0821ef55332a64cc0724491279000e588c6da64ea5f2bf4f" I0413 13:04:55.346610 1 controller.go:213] Skipping VolumeAttachment csi-b7952033dfdb180b0821ef55332a64cc0724491279000e588c6da64ea5f2bf4f for attacher csi-mock-csi-mock-volumes-7885 I0413 13:04:55.346604 1 controller.go:198] Started VA processing "csi-e20a74cbbf46188afea9cf90c3fee98da604ca9b7c56c4bd0d6fe786e4573268" I0413 13:04:55.346653 1 controller.go:213] Skipping VolumeAttachment csi-e20a74cbbf46188afea9cf90c3fee98da604ca9b7c56c4bd0d6fe786e4573268 for attacher csi-mock-csi-mock-volumes-6849 I0413 13:04:55.346727 1 controller.go:198] Started VA processing "csi-8d8b5e9bde507186f7bdb7bd0f3fb22f8d98d379ccf3f895b4d4116373be7781" I0413 13:04:55.346828 1 controller.go:213] Skipping VolumeAttachment csi-8d8b5e9bde507186f7bdb7bd0f3fb22f8d98d379ccf3f895b4d4116373be7781 for attacher csi-mock-csi-mock-volumes-7645 I0413 13:04:55.346586 1 controller.go:198] Started VA processing "csi-3ecb3d617d78d1463db05c0c158e7e547a4750d62ee852055ece2663815d3afc" I0413 13:04:55.346871 1 controller.go:213] Skipping VolumeAttachment csi-3ecb3d617d78d1463db05c0c158e7e547a4750d62ee852055ece2663815d3afc for attacher csi-mock-csi-mock-volumes-4309 I0413 13:04:55.346605 1 connection.go:183] GRPC request: {} I0413 13:04:55.350065 1 connection.go:185] GRPC response: {"entries":[{"status":{"published_node_ids":["csi-mock-csi-mock-volumes-8039"]},"volume":{"capacity_bytes":107374182400,"volume_context":{"name":"Mock Volume 1"},"volume_id":"1"}},{"status":{"published_node_ids":["csi-mock-csi-mock-volumes-8039"]},"volume":{"capacity_bytes":107374182400,"volume_context":{"name":"Mock Volume 2"},"volume_id":"2"}},{"status":{"published_node_ids":["csi-mock-csi-mock-volumes-8039"]},"volume":{"capacity_bytes":107374182400,"volume_context":{"name":"Mock Volume 3"},"volume_id":"3"}}]} I0413 13:04:55.360028 1 connection.go:186] GRPC error: W0413 13:04:55.360066 1 csi_handler.go:159] Failed to repair volume handle for driver csi-mock-csi-mock-volumes-4309: could not find In-Tree driver name for CSI plugin csi-mock-csi-mock-volumes-4309 W0413 13:04:55.360081 1 csi_handler.go:159] Failed to repair volume handle for driver csi-mock-csi-mock-volumes-7885: could not find In-Tree driver name for CSI plugin csi-mock-csi-mock-volumes-7885 W0413 13:04:55.360092 1 csi_handler.go:159] Failed to repair volume handle for driver csi-mock-csi-mock-volumes-6849: could not find In-Tree driver name for CSI plugin csi-mock-csi-mock-volumes-6849 W0413 13:04:55.360102 1 csi_handler.go:159] Failed to repair volume handle for driver csi-mock-csi-mock-volumes-7645: could not find In-Tree driver name for CSI plugin csi-mock-csi-mock-volumes-7645 I0413 13:05:00.395771 1 controller.go:198] Started VA processing "csi-4e8fd8451bb5f6d1a2d381136b9f2376713ac66e35e3add56615c192101c368c" I0413 13:05:00.395805 1 csi_handler.go:209] CSIHandler: processing VA "csi-4e8fd8451bb5f6d1a2d381136b9f2376713ac66e35e3add56615c192101c368c" I0413 13:05:00.395816 1 csi_handler.go:236] Attaching "csi-4e8fd8451bb5f6d1a2d381136b9f2376713ac66e35e3add56615c192101c368c" I0413 13:05:00.395828 1 csi_handler.go:396] Starting attach operation for "csi-4e8fd8451bb5f6d1a2d381136b9f2376713ac66e35e3add56615c192101c368c" I0413 13:05:00.395920 1 csi_handler.go:335] Adding finalizer to PV "pvc-8354f2c8-9502-4af7-89d9-aa9e341d5906" I0413 13:05:00.403116 1 csi_handler.go:344] PV finalizer added to "pvc-8354f2c8-9502-4af7-89d9-aa9e341d5906" I0413 13:05:00.403170 1 csi_handler.go:689] Found NodeID csi-mock-csi-mock-volumes-8039 in CSINode leguer-worker2 I0413 13:05:00.403245 1 csi_handler.go:297] VA finalizer added to "csi-4e8fd8451bb5f6d1a2d381136b9f2376713ac66e35e3add56615c192101c368c" I0413 13:05:00.403268 1 csi_handler.go:311] NodeID annotation added to "csi-4e8fd8451bb5f6d1a2d381136b9f2376713ac66e35e3add56615c192101c368c" I0413 13:05:00.428930 1 connection.go:182] GRPC call: /csi.v1.Controller/ControllerPublishVolume I0413 13:05:00.428966 1 connection.go:183] GRPC request: {"node_id":"csi-mock-csi-mock-volumes-8039","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":1}},"volume_context":{"name":"pvc-8354f2c8-9502-4af7-89d9-aa9e341d5906","storage.kubernetes.io/csiProvisionerIdentity":"1618319095284-8081-csi-mock-csi-mock-volumes-8039"},"volume_id":"4"} I0413 13:05:00.433234 1 connection.go:185] GRPC response: {"publish_context":{"device":"/dev/mock","readonly":"false"}} I0413 13:05:00.434218 1 connection.go:186] GRPC error: I0413 13:05:00.434233 1 csi_handler.go:249] Attached "csi-4e8fd8451bb5f6d1a2d381136b9f2376713ac66e35e3add56615c192101c368c" I0413 13:05:00.434251 1 util.go:35] Marking as attached "csi-4e8fd8451bb5f6d1a2d381136b9f2376713ac66e35e3add56615c192101c368c" I0413 13:05:00.476344 1 util.go:48] Marked as attached "csi-4e8fd8451bb5f6d1a2d381136b9f2376713ac66e35e3add56615c192101c368c" I0413 13:05:00.476397 1 csi_handler.go:255] Fully attached "csi-4e8fd8451bb5f6d1a2d381136b9f2376713ac66e35e3add56615c192101c368c" I0413 13:05:00.476426 1 csi_handler.go:225] CSIHandler: finished processing "csi-4e8fd8451bb5f6d1a2d381136b9f2376713ac66e35e3add56615c192101c368c" I0413 13:05:00.476472 1 controller.go:198] Started VA processing "csi-4e8fd8451bb5f6d1a2d381136b9f2376713ac66e35e3add56615c192101c368c" I0413 13:05:00.476502 1 csi_handler.go:209] CSIHandler: processing VA "csi-4e8fd8451bb5f6d1a2d381136b9f2376713ac66e35e3add56615c192101c368c" I0413 13:05:00.476522 1 csi_handler.go:231] "csi-4e8fd8451bb5f6d1a2d381136b9f2376713ac66e35e3add56615c192101c368c" is already attached I0413 13:05:00.476536 1 csi_handler.go:225] CSIHandler: finished processing "csi-4e8fd8451bb5f6d1a2d381136b9f2376713ac66e35e3add56615c192101c368c" I0413 13:05:55.360376 1 csi_handler.go:116] Reconciling VolumeAttachments with driver backend state I0413 13:05:55.360451 1 connection.go:182] GRPC call: /csi.v1.Controller/ListVolumes I0413 13:05:55.360463 1 connection.go:183] GRPC request: {} I0413 13:05:55.362559 1 connection.go:185] GRPC response: {"entries":[{"status":{"published_node_ids":["csi-mock-csi-mock-volumes-8039"]},"volume":{"capacity_bytes":107374182400,"volume_context":{"name":"Mock Volume 1"},"volume_id":"1"}},{"status":{"published_node_ids":["csi-mock-csi-mock-volumes-8039"]},"volume":{"capacity_bytes":107374182400,"volume_context":{"name":"Mock Volume 2"},"volume_id":"2"}},{"status":{"published_node_ids":["csi-mock-csi-mock-volumes-8039"]},"volume":{"capacity_bytes":107374182400,"volume_context":{"name":"Mock Volume 3"},"volume_id":"3"}},{"status":{"published_node_ids":["csi-mock-csi-mock-volumes-8039"]},"volume":{"capacity_bytes":1073741824,"volume_context":{"csi-mock-csi-mock-volumes-8039/dev":"/dev/mock","csi-mock-csi-mock-volumes-8039/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-8354f2c8-9502-4af7-89d9-aa9e341d5906/globalmount":"/dev/mock","csi-mock-csi-mock-volumes-8039/var/lib/kubelet/pods/6960a010-3b1a-4cb2-bf50-d6d020730b01/volumes/kubernetes.io~csi/pvc-8354f2c8-9502-4af7-89d9-aa9e341d5906/mount":"/var/lib/kubelet/pods/6960a010-3b1a-4cb2-bf50-d6d020730b01/volumes/kubernetes.io~csi/pvc-8354f2c8-9502-4af7-89d9-aa9e341d5906/mount","name":"pvc-8354f2c8-9502-4af7-89d9-aa9e341d5906","readonly":"false"},"volume_id":"4"}}]} I0413 13:05:55.373299 1 connection.go:186] GRPC error: W0413 13:05:55.373320 1 csi_handler.go:159] Failed to repair volume handle for driver csi-mock-csi-mock-volumes-4309: could not find In-Tree driver name for CSI plugin csi-mock-csi-mock-volumes-4309 W0413 13:05:55.373333 1 csi_handler.go:159] Failed to repair volume handle for driver csi-mock-csi-mock-volumes-7885: could not find In-Tree driver name for CSI plugin csi-mock-csi-mock-volumes-7885 W0413 13:05:55.373339 1 csi_handler.go:159] Failed to repair volume handle for driver csi-mock-csi-mock-volumes-6849: could not find In-Tree driver name for CSI plugin csi-mock-csi-mock-volumes-6849 W0413 13:05:55.373345 1 csi_handler.go:159] Failed to repair volume handle for driver csi-mock-csi-mock-volumes-7645: could not find In-Tree driver name for CSI plugin csi-mock-csi-mock-volumes-7645 W0413 13:05:55.373351 1 csi_handler.go:159] Failed to repair volume handle for driver csi-mock-csi-mock-volumes-8039: could not find In-Tree driver name for CSI plugin csi-mock-csi-mock-volumes-8039 I0413 13:06:32.082714 1 controller.go:198] Started VA processing "csi-06c4824bbd98d5daec60d555501c03091a208101f4e15cb4906666f69b6f3754" I0413 13:06:32.082754 1 controller.go:213] Skipping VolumeAttachment csi-06c4824bbd98d5daec60d555501c03091a208101f4e15cb4906666f69b6f3754 for attacher csi-mock-csi-mock-volumes-169 I0413 13:06:32.267672 1 controller.go:198] Started VA processing "csi-06c4824bbd98d5daec60d555501c03091a208101f4e15cb4906666f69b6f3754" I0413 13:06:32.267737 1 controller.go:213] Skipping VolumeAttachment csi-06c4824bbd98d5daec60d555501c03091a208101f4e15cb4906666f69b6f3754 for attacher csi-mock-csi-mock-volumes-169 I0413 13:06:32.304078 1 controller.go:198] Started VA processing "csi-06c4824bbd98d5daec60d555501c03091a208101f4e15cb4906666f69b6f3754" I0413 13:06:32.304137 1 controller.go:213] Skipping VolumeAttachment csi-06c4824bbd98d5daec60d555501c03091a208101f4e15cb4906666f69b6f3754 for attacher csi-mock-csi-mock-volumes-169 I0413 13:06:55.373588 1 csi_handler.go:116] Reconciling VolumeAttachments with driver backend state I0413 13:06:55.373677 1 connection.go:182] GRPC call: /csi.v1.Controller/ListVolumes I0413 13:06:55.373697 1 connection.go:183] GRPC request: {} I0413 13:06:55.375703 1 connection.go:185] GRPC response: {"entries":[{"status":{"published_node_ids":["csi-mock-csi-mock-volumes-8039"]},"volume":{"capacity_bytes":107374182400,"volume_context":{"name":"Mock Volume 1"},"volume_id":"1"}},{"status":{"published_node_ids":["csi-mock-csi-mock-volumes-8039"]},"volume":{"capacity_bytes":107374182400,"volume_context":{"name":"Mock Volume 2"},"volume_id":"2"}},{"status":{"published_node_ids":["csi-mock-csi-mock-volumes-8039"]},"volume":{"capacity_bytes":107374182400,"volume_context":{"name":"Mock Volume 3"},"volume_id":"3"}},{"status":{"published_node_ids":["csi-mock-csi-mock-volumes-8039"]},"volume":{"capacity_bytes":1073741824,"volume_context":{"csi-mock-csi-mock-volumes-8039/dev":"/dev/mock","csi-mock-csi-mock-volumes-8039/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-8354f2c8-9502-4af7-89d9-aa9e341d5906/globalmount":"/dev/mock","csi-mock-csi-mock-volumes-8039/var/lib/kubelet/pods/6960a010-3b1a-4cb2-bf50-d6d020730b01/volumes/kubernetes.io~csi/pvc-8354f2c8-9502-4af7-89d9-aa9e341d5906/mount":"/var/lib/kubelet/pods/6960a010-3b1a-4cb2-bf50-d6d020730b01/volumes/kubernetes.io~csi/pvc-8354f2c8-9502-4af7-89d9-aa9e341d5906/mount","name":"pvc-8354f2c8-9502-4af7-89d9-aa9e341d5906","readonly":"false"},"volume_id":"4"}}]} I0413 13:06:55.387024 1 connection.go:186] GRPC error: W0413 13:06:55.387057 1 csi_handler.go:159] Failed to repair volume handle for driver csi-mock-csi-mock-volumes-6849: could not find In-Tree driver name for CSI plugin csi-mock-csi-mock-volumes-6849 W0413 13:06:55.387089 1 csi_handler.go:159] Failed to repair volume handle for driver csi-mock-csi-mock-volumes-7645: could not find In-Tree driver name for CSI plugin csi-mock-csi-mock-volumes-7645 W0413 13:06:55.387115 1 csi_handler.go:159] Failed to repair volume handle for driver csi-mock-csi-mock-volumes-8039: could not find In-Tree driver name for CSI plugin csi-mock-csi-mock-volumes-8039 W0413 13:06:55.387132 1 csi_handler.go:159] Failed to repair volume handle for driver csi-mock-csi-mock-volumes-169: could not find In-Tree driver name for CSI plugin csi-mock-csi-mock-volumes-169 W0413 13:06:55.387147 1 csi_handler.go:159] Failed to repair volume handle for driver csi-mock-csi-mock-volumes-4309: could not find In-Tree driver name for CSI plugin csi-mock-csi-mock-volumes-4309 W0413 13:06:55.387160 1 csi_handler.go:159] Failed to repair volume handle for driver csi-mock-csi-mock-volumes-7885: could not find In-Tree driver name for CSI plugin csi-mock-csi-mock-volumes-7885 I0413 13:07:55.387407 1 csi_handler.go:116] Reconciling VolumeAttachments with driver backend state I0413 13:07:55.387518 1 connection.go:182] GRPC call: /csi.v1.Controller/ListVolumes I0413 13:07:55.387532 1 connection.go:183] GRPC request: {} I0413 13:07:55.389870 1 connection.go:185] GRPC response: {"entries":[{"status":{"published_node_ids":["csi-mock-csi-mock-volumes-8039"]},"volume":{"capacity_bytes":107374182400,"volume_context":{"name":"Mock Volume 1"},"volume_id":"1"}},{"status":{"published_node_ids":["csi-mock-csi-mock-volumes-8039"]},"volume":{"capacity_bytes":107374182400,"volume_context":{"name":"Mock Volume 2"},"volume_id":"2"}},{"status":{"published_node_ids":["csi-mock-csi-mock-volumes-8039"]},"volume":{"capacity_bytes":107374182400,"volume_context":{"name":"Mock Volume 3"},"volume_id":"3"}},{"status":{"published_node_ids":["csi-mock-csi-mock-volumes-8039"]},"volume":{"capacity_bytes":1073741824,"volume_context":{"csi-mock-csi-mock-volumes-8039/dev":"/dev/mock","csi-mock-csi-mock-volumes-8039/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-8354f2c8-9502-4af7-89d9-aa9e341d5906/globalmount":"/dev/mock","csi-mock-csi-mock-volumes-8039/var/lib/kubelet/pods/6960a010-3b1a-4cb2-bf50-d6d020730b01/volumes/kubernetes.io~csi/pvc-8354f2c8-9502-4af7-89d9-aa9e341d5906/mount":"/var/lib/kubelet/pods/6960a010-3b1a-4cb2-bf50-d6d020730b01/volumes/kubernetes.io~csi/pvc-8354f2c8-9502-4af7-89d9-aa9e341d5906/mount","name":"pvc-8354f2c8-9502-4af7-89d9-aa9e341d5906","readonly":"false"},"volume_id":"4"}}]} I0413 13:07:55.398875 1 connection.go:186] GRPC error: W0413 13:07:55.398899 1 csi_handler.go:159] Failed to repair volume handle for driver csi-mock-csi-mock-volumes-4309: could not find In-Tree driver name for CSI plugin csi-mock-csi-mock-volumes-4309 W0413 13:07:55.398912 1 csi_handler.go:159] Failed to repair volume handle for driver csi-mock-csi-mock-volumes-7885: could not find In-Tree driver name for CSI plugin csi-mock-csi-mock-volumes-7885 W0413 13:07:55.398920 1 csi_handler.go:159] Failed to repair volume handle for driver csi-mock-csi-mock-volumes-6849: could not find In-Tree driver name for CSI plugin csi-mock-csi-mock-volumes-6849 W0413 13:07:55.398929 1 csi_handler.go:159] Failed to repair volume handle for driver csi-mock-csi-mock-volumes-7645: could not find In-Tree driver name for CSI plugin csi-mock-csi-mock-volumes-7645 W0413 13:07:55.398937 1 csi_handler.go:159] Failed to repair volume handle for driver csi-mock-csi-mock-volumes-8039: could not find In-Tree driver name for CSI plugin csi-mock-csi-mock-volumes-8039 W0413 13:07:55.398946 1 csi_handler.go:159] Failed to repair volume handle for driver csi-mock-csi-mock-volumes-169: could not find In-Tree driver name for CSI plugin csi-mock-csi-mock-volumes-169 I0413 13:08:15.086364 1 controller.go:198] Started VA processing "csi-8d8b5e9bde507186f7bdb7bd0f3fb22f8d98d379ccf3f895b4d4116373be7781" I0413 13:08:15.086412 1 controller.go:213] Skipping VolumeAttachment csi-8d8b5e9bde507186f7bdb7bd0f3fb22f8d98d379ccf3f895b4d4116373be7781 for attacher csi-mock-csi-mock-volumes-7645 I0413 13:08:15.098651 1 controller.go:232] Started PV processing "pvc-801363df-e890-4451-8ea0-35d9cfad5df8" I0413 13:08:15.098696 1 csi_handler.go:591] CSIHandler: processing PV "pvc-801363df-e890-4451-8ea0-35d9cfad5df8" I0413 13:08:15.098713 1 csi_handler.go:595] CSIHandler: processing PV "pvc-801363df-e890-4451-8ea0-35d9cfad5df8": no deletion timestamp, ignoring I0413 13:08:19.670086 1 controller.go:198] Started VA processing "csi-bc119acc7356290427580203af65031aed94948b57933d06c25bcc48963f7737" I0413 13:08:19.670126 1 controller.go:213] Skipping VolumeAttachment csi-bc119acc7356290427580203af65031aed94948b57933d06c25bcc48963f7737 for attacher csi-mock-csi-mock-volumes-6946 I0413 13:08:19.718858 1 controller.go:198] Started VA processing "csi-bc119acc7356290427580203af65031aed94948b57933d06c25bcc48963f7737" I0413 13:08:19.719100 1 controller.go:213] Skipping VolumeAttachment csi-bc119acc7356290427580203af65031aed94948b57933d06c25bcc48963f7737 for attacher csi-mock-csi-mock-volumes-6946 I0413 13:08:19.729590 1 controller.go:198] Started VA processing "csi-bc119acc7356290427580203af65031aed94948b57933d06c25bcc48963f7737" I0413 13:08:19.729615 1 controller.go:213] Skipping VolumeAttachment csi-bc119acc7356290427580203af65031aed94948b57933d06c25bcc48963f7737 for attacher csi-mock-csi-mock-volumes-6946 I0413 13:08:25.094391 1 controller.go:198] Started VA processing "csi-3ecb3d617d78d1463db05c0c158e7e547a4750d62ee852055ece2663815d3afc" I0413 13:08:25.094441 1 controller.go:213] Skipping VolumeAttachment csi-3ecb3d617d78d1463db05c0c158e7e547a4750d62ee852055ece2663815d3afc for attacher csi-mock-csi-mock-volumes-4309 I0413 13:08:25.167296 1 controller.go:232] Started PV processing "pvc-2b94d08a-b215-443e-9676-de1f2f1547a9" I0413 13:08:25.167325 1 csi_handler.go:591] CSIHandler: processing PV "pvc-2b94d08a-b215-443e-9676-de1f2f1547a9" I0413 13:08:25.167339 1 csi_handler.go:595] CSIHandler: processing PV "pvc-2b94d08a-b215-443e-9676-de1f2f1547a9": no deletion timestamp, ignoring I0413 13:08:35.457916 1 controller.go:198] Started VA processing "csi-b7952033dfdb180b0821ef55332a64cc0724491279000e588c6da64ea5f2bf4f" I0413 13:08:35.457958 1 controller.go:213] Skipping VolumeAttachment csi-b7952033dfdb180b0821ef55332a64cc0724491279000e588c6da64ea5f2bf4f for attacher csi-mock-csi-mock-volumes-7885 I0413 13:08:35.539239 1 controller.go:232] Started PV processing "pvc-05c19cec-1410-4a0d-93ea-03c9f4b5e5ab" I0413 13:08:35.539296 1 csi_handler.go:591] CSIHandler: processing PV "pvc-05c19cec-1410-4a0d-93ea-03c9f4b5e5ab" I0413 13:08:35.539312 1 csi_handler.go:595] CSIHandler: processing PV "pvc-05c19cec-1410-4a0d-93ea-03c9f4b5e5ab": no deletion timestamp, ignoring I0413 13:08:55.399145 1 csi_handler.go:116] Reconciling VolumeAttachments with driver backend state I0413 13:08:55.399219 1 connection.go:182] GRPC call: /csi.v1.Controller/ListVolumes I0413 13:08:55.399230 1 connection.go:183] GRPC request: {} I0413 13:08:55.401066 1 connection.go:185] GRPC response: {"entries":[{"status":{"published_node_ids":["csi-mock-csi-mock-volumes-8039"]},"volume":{"capacity_bytes":107374182400,"volume_context":{"name":"Mock Volume 1"},"volume_id":"1"}},{"status":{"published_node_ids":["csi-mock-csi-mock-volumes-8039"]},"volume":{"capacity_bytes":107374182400,"volume_context":{"name":"Mock Volume 2"},"volume_id":"2"}},{"status":{"published_node_ids":["csi-mock-csi-mock-volumes-8039"]},"volume":{"capacity_bytes":107374182400,"volume_context":{"name":"Mock Volume 3"},"volume_id":"3"}},{"status":{"published_node_ids":["csi-mock-csi-mock-volumes-8039"]},"volume":{"capacity_bytes":1073741824,"volume_context":{"csi-mock-csi-mock-volumes-8039/dev":"/dev/mock","csi-mock-csi-mock-volumes-8039/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-8354f2c8-9502-4af7-89d9-aa9e341d5906/globalmount":"/dev/mock","csi-mock-csi-mock-volumes-8039/var/lib/kubelet/pods/6960a010-3b1a-4cb2-bf50-d6d020730b01/volumes/kubernetes.io~csi/pvc-8354f2c8-9502-4af7-89d9-aa9e341d5906/mount":"/var/lib/kubelet/pods/6960a010-3b1a-4cb2-bf50-d6d020730b01/volumes/kubernetes.io~csi/pvc-8354f2c8-9502-4af7-89d9-aa9e341d5906/mount","name":"pvc-8354f2c8-9502-4af7-89d9-aa9e341d5906","readonly":"false"},"volume_id":"4"}}]} I0413 13:08:55.409368 1 connection.go:186] GRPC error: W0413 13:08:55.409391 1 csi_handler.go:159] Failed to repair volume handle for driver csi-mock-csi-mock-volumes-6849: could not find In-Tree driver name for CSI plugin csi-mock-csi-mock-volumes-6849 W0413 13:08:55.409402 1 csi_handler.go:159] Failed to repair volume handle for driver csi-mock-csi-mock-volumes-6946: could not find In-Tree driver name for CSI plugin csi-mock-csi-mock-volumes-6946 W0413 13:08:55.409409 1 csi_handler.go:159] Failed to repair volume handle for driver csi-mock-csi-mock-volumes-8039: could not find In-Tree driver name for CSI plugin csi-mock-csi-mock-volumes-8039 W0413 13:08:55.409414 1 csi_handler.go:159] Failed to repair volume handle for driver csi-mock-csi-mock-volumes-169: could not find In-Tree driver name for CSI plugin csi-mock-csi-mock-volumes-169 I0413 13:09:55.409667 1 csi_handler.go:116] Reconciling VolumeAttachments with driver backend state I0413 13:09:55.409769 1 connection.go:182] GRPC call: /csi.v1.Controller/ListVolumes I0413 13:09:55.409782 1 connection.go:183] GRPC request: {} I0413 13:09:55.411635 1 connection.go:185] GRPC response: {"entries":[{"status":{"published_node_ids":["csi-mock-csi-mock-volumes-8039"]},"volume":{"capacity_bytes":107374182400,"volume_context":{"name":"Mock Volume 1"},"volume_id":"1"}},{"status":{"published_node_ids":["csi-mock-csi-mock-volumes-8039"]},"volume":{"capacity_bytes":107374182400,"volume_context":{"name":"Mock Volume 2"},"volume_id":"2"}},{"status":{"published_node_ids":["csi-mock-csi-mock-volumes-8039"]},"volume":{"capacity_bytes":107374182400,"volume_context":{"name":"Mock Volume 3"},"volume_id":"3"}},{"status":{"published_node_ids":["csi-mock-csi-mock-volumes-8039"]},"volume":{"capacity_bytes":1073741824,"volume_context":{"csi-mock-csi-mock-volumes-8039/dev":"/dev/mock","csi-mock-csi-mock-volumes-8039/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-8354f2c8-9502-4af7-89d9-aa9e341d5906/globalmount":"/dev/mock","csi-mock-csi-mock-volumes-8039/var/lib/kubelet/pods/6960a010-3b1a-4cb2-bf50-d6d020730b01/volumes/kubernetes.io~csi/pvc-8354f2c8-9502-4af7-89d9-aa9e341d5906/mount":"/var/lib/kubelet/pods/6960a010-3b1a-4cb2-bf50-d6d020730b01/volumes/kubernetes.io~csi/pvc-8354f2c8-9502-4af7-89d9-aa9e341d5906/mount","name":"pvc-8354f2c8-9502-4af7-89d9-aa9e341d5906","readonly":"false"},"volume_id":"4"}}]} I0413 13:09:55.421476 1 connection.go:186] GRPC error: W0413 13:09:55.421505 1 csi_handler.go:159] Failed to repair volume handle for driver csi-mock-csi-mock-volumes-6849: could not find In-Tree driver name for CSI plugin csi-mock-csi-mock-volumes-6849 W0413 13:09:55.421533 1 csi_handler.go:159] Failed to repair volume handle for driver csi-mock-csi-mock-volumes-6946: could not find In-Tree driver name for CSI plugin csi-mock-csi-mock-volumes-6946 W0413 13:09:55.421545 1 csi_handler.go:159] Failed to repair volume handle for driver csi-mock-csi-mock-volumes-8039: could not find In-Tree driver name for CSI plugin csi-mock-csi-mock-volumes-8039 W0413 13:09:55.421557 1 csi_handler.go:159] Failed to repair volume handle for driver csi-mock-csi-mock-volumes-169: could not find In-Tree driver name for CSI plugin csi-mock-csi-mock-volumes-169 I0413 13:09:55.774027 1 controller.go:198] Started VA processing "csi-e20a74cbbf46188afea9cf90c3fee98da604ca9b7c56c4bd0d6fe786e4573268" I0413 13:09:55.774072 1 controller.go:213] Skipping VolumeAttachment csi-e20a74cbbf46188afea9cf90c3fee98da604ca9b7c56c4bd0d6fe786e4573268 for attacher csi-mock-csi-mock-volumes-6849 I0413 13:09:56.083332 1 controller.go:232] Started PV processing "pvc-3c783a59-06e1-4a27-bf2c-33802d323818" I0413 13:09:56.083378 1 csi_handler.go:591] CSIHandler: processing PV "pvc-3c783a59-06e1-4a27-bf2c-33802d323818" I0413 13:09:56.083399 1 csi_handler.go:595] CSIHandler: processing PV "pvc-3c783a59-06e1-4a27-bf2c-33802d323818": no deletion timestamp, ignoring I0413 13:10:05.375710 1 controller.go:198] Started VA processing "csi-4e8fd8451bb5f6d1a2d381136b9f2376713ac66e35e3add56615c192101c368c" I0413 13:10:05.375794 1 csi_handler.go:209] CSIHandler: processing VA "csi-4e8fd8451bb5f6d1a2d381136b9f2376713ac66e35e3add56615c192101c368c" I0413 13:10:05.375816 1 csi_handler.go:260] Starting detach operation for "csi-4e8fd8451bb5f6d1a2d381136b9f2376713ac66e35e3add56615c192101c368c" I0413 13:10:05.375977 1 csi_handler.go:267] Detaching "csi-4e8fd8451bb5f6d1a2d381136b9f2376713ac66e35e3add56615c192101c368c" I0413 13:10:05.376034 1 csi_handler.go:689] Found NodeID csi-mock-csi-mock-volumes-8039 in CSINode leguer-worker2 I0413 13:10:05.376128 1 connection.go:182] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume I0413 13:10:05.376146 1 connection.go:183] GRPC request: {"node_id":"csi-mock-csi-mock-volumes-8039","volume_id":"4"} I0413 13:10:05.378853 1 connection.go:185] GRPC response: {} I0413 13:10:05.379595 1 connection.go:186] GRPC error: I0413 13:10:05.379616 1 csi_handler.go:549] Detached "csi-4e8fd8451bb5f6d1a2d381136b9f2376713ac66e35e3add56615c192101c368c" I0413 13:10:05.379679 1 util.go:76] Marking as detached "csi-4e8fd8451bb5f6d1a2d381136b9f2376713ac66e35e3add56615c192101c368c" I0413 13:10:05.426743 1 controller.go:232] Started PV processing "pvc-8354f2c8-9502-4af7-89d9-aa9e341d5906" I0413 13:10:05.426804 1 csi_handler.go:591] CSIHandler: processing PV "pvc-8354f2c8-9502-4af7-89d9-aa9e341d5906" I0413 13:10:05.426845 1 csi_handler.go:595] CSIHandler: processing PV "pvc-8354f2c8-9502-4af7-89d9-aa9e341d5906": no deletion timestamp, ignoring I0413 13:10:05.427200 1 util.go:90] Finalizer removed from "csi-4e8fd8451bb5f6d1a2d381136b9f2376713ac66e35e3add56615c192101c368c" I0413 13:10:05.427256 1 csi_handler.go:280] Fully detached "csi-4e8fd8451bb5f6d1a2d381136b9f2376713ac66e35e3add56615c192101c368c" I0413 13:10:05.427282 1 csi_handler.go:225] CSIHandler: finished processing "csi-4e8fd8451bb5f6d1a2d381136b9f2376713ac66e35e3add56615c192101c368c" I0413 13:10:12.495948 1 controller.go:232] Started PV processing "pvc-8354f2c8-9502-4af7-89d9-aa9e341d5906" I0413 13:10:12.496002 1 csi_handler.go:591] CSIHandler: processing PV "pvc-8354f2c8-9502-4af7-89d9-aa9e341d5906" I0413 13:10:12.496040 1 csi_handler.go:633] CSIHandler: processing PV "pvc-8354f2c8-9502-4af7-89d9-aa9e341d5906": no VA found, removing finalizer E0413 13:10:12.505111 1 csi_handler.go:650] Failed to remove finalizer from PV "pvc-8354f2c8-9502-4af7-89d9-aa9e341d5906": PersistentVolume "pvc-8354f2c8-9502-4af7-89d9-aa9e341d5906" is invalid: metadata.finalizers: Forbidden: no new finalizers can be added if the object is being deleted, found new finalizers []string{"kubernetes.io/pv-protection"} I0413 13:10:12.505190 1 controller.go:232] Started PV processing "pvc-8354f2c8-9502-4af7-89d9-aa9e341d5906" I0413 13:10:12.505211 1 csi_handler.go:591] CSIHandler: processing PV "pvc-8354f2c8-9502-4af7-89d9-aa9e341d5906" I0413 13:10:12.505243 1 csi_handler.go:633] CSIHandler: processing PV "pvc-8354f2c8-9502-4af7-89d9-aa9e341d5906": no VA found, removing finalizer I0413 13:10:12.548081 1 csi_handler.go:655] Removed finalizer from PV "pvc-8354f2c8-9502-4af7-89d9-aa9e341d5906" I0413 13:10:13.505394 1 controller.go:232] Started PV processing "pvc-8354f2c8-9502-4af7-89d9-aa9e341d5906" I0413 13:10:13.505441 1 controller.go:239] PV "pvc-8354f2c8-9502-4af7-89d9-aa9e341d5906" deleted, ignoring ==== end of pod log for container csi-mockplugin-attacher-0/csi-attacher ====