==== start of pod log for container csi-mockplugin-attacher-0/csi-attacher ==== I0618 00:12:40.458588 1 main.go:91] Version: v2.2.0-0-g97411fa7 I0618 00:12:40.459679 1 connection.go:153] Connecting to unix:///csi/csi.sock I0618 00:12:42.405338 1 common.go:111] Probing CSI driver for readiness I0618 00:12:42.405358 1 connection.go:182] GRPC call: /csi.v1.Identity/Probe I0618 00:12:42.405363 1 connection.go:183] GRPC request: {} I0618 00:12:42.406910 1 connection.go:185] GRPC response: {"ready":{"value":true}} I0618 00:12:42.408834 1 connection.go:186] GRPC error: I0618 00:12:42.408869 1 connection.go:182] GRPC call: /csi.v1.Identity/GetPluginInfo I0618 00:12:42.408873 1 connection.go:183] GRPC request: {} I0618 00:12:42.410733 1 connection.go:185] GRPC response: {"manifest":{"url":"https://github.com/kubernetes-csi/csi-test/mock"},"name":"csi-mock-csi-mock-volumes-6733","vendor_version":"0.3.0"} I0618 00:12:42.411043 1 connection.go:186] GRPC error: I0618 00:12:42.411050 1 main.go:136] CSI driver name: "csi-mock-csi-mock-volumes-6733" W0618 00:12:42.411057 1 metrics.go:142] metrics endpoint will not be started because `metrics-address` was not specified. I0618 00:12:42.411061 1 connection.go:182] GRPC call: /csi.v1.Identity/GetPluginCapabilities I0618 00:12:42.411065 1 connection.go:183] GRPC request: {} I0618 00:12:42.411663 1 connection.go:185] GRPC response: {"capabilities":[{"Type":{"Service":{"type":1}}},{"Type":{"VolumeExpansion":{"type":1}}}]} I0618 00:12:42.412489 1 connection.go:186] GRPC error: I0618 00:12:42.412495 1 connection.go:182] GRPC call: /csi.v1.Controller/ControllerGetCapabilities I0618 00:12:42.412499 1 connection.go:183] GRPC request: {} I0618 00:12:42.415772 1 connection.go:185] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":3}}},{"Type":{"Rpc":{"type":10}}},{"Type":{"Rpc":{"type":4}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":8}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":12}}},{"Type":{"Rpc":{"type":11}}},{"Type":{"Rpc":{"type":2}}},{"Type":{"Rpc":{"type":9}}}]} I0618 00:12:42.423930 1 connection.go:186] GRPC error: I0618 00:12:42.423984 1 main.go:162] CSI driver supports ControllerPublishUnpublish, using real CSI handler I0618 00:12:42.423989 1 connection.go:182] GRPC call: /csi.v1.Controller/ControllerGetCapabilities I0618 00:12:42.423993 1 connection.go:183] GRPC request: {} I0618 00:12:42.425096 1 connection.go:185] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":3}}},{"Type":{"Rpc":{"type":10}}},{"Type":{"Rpc":{"type":4}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":8}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":12}}},{"Type":{"Rpc":{"type":11}}},{"Type":{"Rpc":{"type":2}}},{"Type":{"Rpc":{"type":9}}}]} I0618 00:12:42.428168 1 connection.go:186] GRPC error: I0618 00:12:42.428179 1 main.go:175] CSI driver supports list volumes published nodes. Using capability to reconcile volume attachment objects with actual backend state I0618 00:12:42.428266 1 controller.go:121] Starting CSI attacher I0618 00:12:42.428452 1 reflector.go:153] Starting reflector *v1.PersistentVolume (10m0s) from k8s.io/client-go/informers/factory.go:135 I0618 00:12:42.428463 1 reflector.go:188] Listing and watching *v1.PersistentVolume from k8s.io/client-go/informers/factory.go:135 I0618 00:12:42.428997 1 reflector.go:153] Starting reflector *v1beta1.VolumeAttachment (10m0s) from k8s.io/client-go/informers/factory.go:135 I0618 00:12:42.429004 1 reflector.go:188] Listing and watching *v1beta1.VolumeAttachment from k8s.io/client-go/informers/factory.go:135 I0618 00:12:42.429213 1 reflector.go:153] Starting reflector *v1beta1.CSINode (10m0s) from k8s.io/client-go/informers/factory.go:135 I0618 00:12:42.429219 1 reflector.go:188] Listing and watching *v1beta1.CSINode from k8s.io/client-go/informers/factory.go:135 I0618 00:12:42.528369 1 shared_informer.go:227] caches populated I0618 00:12:42.528424 1 csi_handler.go:116] Reconciling VolumeAttachments with driver backend state I0618 00:12:42.528446 1 connection.go:182] GRPC call: /csi.v1.Controller/ListVolumes I0618 00:12:42.528451 1 connection.go:183] GRPC request: {} I0618 00:12:42.528907 1 controller.go:198] Started VA processing "csi-f6af658de6a4535cf2ffaa4483ffbd73aa88881bb61c6e4ea870f2e0319710ab" I0618 00:12:42.528915 1 controller.go:213] Skipping VolumeAttachment csi-f6af658de6a4535cf2ffaa4483ffbd73aa88881bb61c6e4ea870f2e0319710ab for attacher csi-mock-csi-mock-volumes-4171 I0618 00:12:42.529507 1 connection.go:185] GRPC response: {"entries":[{"status":{"published_node_ids":["csi-mock-csi-mock-volumes-6733"]},"volume":{"capacity_bytes":107374182400,"volume_context":{"name":"Mock Volume 1"},"volume_id":"1"}},{"status":{"published_node_ids":["csi-mock-csi-mock-volumes-6733"]},"volume":{"capacity_bytes":107374182400,"volume_context":{"name":"Mock Volume 2"},"volume_id":"2"}},{"status":{"published_node_ids":["csi-mock-csi-mock-volumes-6733"]},"volume":{"capacity_bytes":107374182400,"volume_context":{"name":"Mock Volume 3"},"volume_id":"3"}}]} I0618 00:12:42.535225 1 connection.go:186] GRPC error: W0618 00:12:42.535245 1 csi_handler.go:159] Failed to repair volume handle for driver csi-mock-csi-mock-volumes-4171: could not find In-Tree driver name for CSI plugin csi-mock-csi-mock-volumes-4171 I0618 00:12:48.641114 1 controller.go:198] Started VA processing "csi-35c2c9ec7d5aed383ea0a5ba5122bb4eec9a4511c579973541a45cc4b0c4b96e" I0618 00:12:48.641127 1 controller.go:213] Skipping VolumeAttachment csi-35c2c9ec7d5aed383ea0a5ba5122bb4eec9a4511c579973541a45cc4b0c4b96e for attacher csi-mock-csi-mock-volumes-4040 I0618 00:12:48.648111 1 controller.go:198] Started VA processing "csi-35c2c9ec7d5aed383ea0a5ba5122bb4eec9a4511c579973541a45cc4b0c4b96e" I0618 00:12:48.648119 1 controller.go:213] Skipping VolumeAttachment csi-35c2c9ec7d5aed383ea0a5ba5122bb4eec9a4511c579973541a45cc4b0c4b96e for attacher csi-mock-csi-mock-volumes-4040 I0618 00:12:48.655235 1 controller.go:198] Started VA processing "csi-35c2c9ec7d5aed383ea0a5ba5122bb4eec9a4511c579973541a45cc4b0c4b96e" I0618 00:12:48.655246 1 controller.go:213] Skipping VolumeAttachment csi-35c2c9ec7d5aed383ea0a5ba5122bb4eec9a4511c579973541a45cc4b0c4b96e for attacher csi-mock-csi-mock-volumes-4040 I0618 00:12:49.139806 1 controller.go:198] Started VA processing "csi-d1cb48f0a673ae4b30d69335218d194fb380ad25a49c4281483d6087004537fe" I0618 00:12:49.139964 1 csi_handler.go:209] CSIHandler: processing VA "csi-d1cb48f0a673ae4b30d69335218d194fb380ad25a49c4281483d6087004537fe" I0618 00:12:49.139972 1 csi_handler.go:236] Attaching "csi-d1cb48f0a673ae4b30d69335218d194fb380ad25a49c4281483d6087004537fe" I0618 00:12:49.139982 1 csi_handler.go:396] Starting attach operation for "csi-d1cb48f0a673ae4b30d69335218d194fb380ad25a49c4281483d6087004537fe" I0618 00:12:49.140068 1 csi_handler.go:335] Adding finalizer to PV "pvc-38229df3-fe87-4b3d-a1ab-65de2ca36a09" I0618 00:12:49.145075 1 csi_handler.go:344] PV finalizer added to "pvc-38229df3-fe87-4b3d-a1ab-65de2ca36a09" I0618 00:12:49.145093 1 csi_handler.go:689] Found NodeID csi-mock-csi-mock-volumes-6733 in CSINode node2 I0618 00:12:49.145109 1 csi_handler.go:297] VA finalizer added to "csi-d1cb48f0a673ae4b30d69335218d194fb380ad25a49c4281483d6087004537fe" I0618 00:12:49.145115 1 csi_handler.go:311] NodeID annotation added to "csi-d1cb48f0a673ae4b30d69335218d194fb380ad25a49c4281483d6087004537fe" I0618 00:12:49.148127 1 connection.go:182] GRPC call: /csi.v1.Controller/ControllerPublishVolume I0618 00:12:49.148133 1 connection.go:183] GRPC request: {"node_id":"csi-mock-csi-mock-volumes-6733","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":1}},"volume_context":{"name":"pvc-38229df3-fe87-4b3d-a1ab-65de2ca36a09","storage.kubernetes.io/csiProvisionerIdentity":"1655511162545-8081-csi-mock-csi-mock-volumes-6733"},"volume_id":"4"} I0618 00:12:49.150400 1 connection.go:185] GRPC response: {"publish_context":{"device":"/dev/mock","readonly":"false"}} I0618 00:12:49.150692 1 connection.go:186] GRPC error: I0618 00:12:49.150698 1 csi_handler.go:249] Attached "csi-d1cb48f0a673ae4b30d69335218d194fb380ad25a49c4281483d6087004537fe" I0618 00:12:49.150703 1 util.go:35] Marking as attached "csi-d1cb48f0a673ae4b30d69335218d194fb380ad25a49c4281483d6087004537fe" I0618 00:12:49.154891 1 util.go:48] Marked as attached "csi-d1cb48f0a673ae4b30d69335218d194fb380ad25a49c4281483d6087004537fe" I0618 00:12:49.154911 1 csi_handler.go:255] Fully attached "csi-d1cb48f0a673ae4b30d69335218d194fb380ad25a49c4281483d6087004537fe" I0618 00:12:49.154917 1 csi_handler.go:225] CSIHandler: finished processing "csi-d1cb48f0a673ae4b30d69335218d194fb380ad25a49c4281483d6087004537fe" I0618 00:12:49.154935 1 controller.go:198] Started VA processing "csi-d1cb48f0a673ae4b30d69335218d194fb380ad25a49c4281483d6087004537fe" I0618 00:12:49.154940 1 csi_handler.go:209] CSIHandler: processing VA "csi-d1cb48f0a673ae4b30d69335218d194fb380ad25a49c4281483d6087004537fe" I0618 00:12:49.154946 1 csi_handler.go:231] "csi-d1cb48f0a673ae4b30d69335218d194fb380ad25a49c4281483d6087004537fe" is already attached I0618 00:12:49.154950 1 csi_handler.go:225] CSIHandler: finished processing "csi-d1cb48f0a673ae4b30d69335218d194fb380ad25a49c4281483d6087004537fe" I0618 00:12:56.402318 1 controller.go:198] Started VA processing "csi-ec78d8c67d63bb8dc9a5cddc8a1cbf890273d62bbaa6ef56df1aff8b52e7dc3d" I0618 00:12:56.402334 1 controller.go:213] Skipping VolumeAttachment csi-ec78d8c67d63bb8dc9a5cddc8a1cbf890273d62bbaa6ef56df1aff8b52e7dc3d for attacher csi-mock-csi-mock-volumes-5292 I0618 00:12:56.416830 1 controller.go:198] Started VA processing "csi-ec78d8c67d63bb8dc9a5cddc8a1cbf890273d62bbaa6ef56df1aff8b52e7dc3d" I0618 00:12:56.416839 1 controller.go:213] Skipping VolumeAttachment csi-ec78d8c67d63bb8dc9a5cddc8a1cbf890273d62bbaa6ef56df1aff8b52e7dc3d for attacher csi-mock-csi-mock-volumes-5292 I0618 00:12:56.460745 1 controller.go:198] Started VA processing "csi-ec78d8c67d63bb8dc9a5cddc8a1cbf890273d62bbaa6ef56df1aff8b52e7dc3d" I0618 00:12:56.460772 1 controller.go:213] Skipping VolumeAttachment csi-ec78d8c67d63bb8dc9a5cddc8a1cbf890273d62bbaa6ef56df1aff8b52e7dc3d for attacher csi-mock-csi-mock-volumes-5292 I0618 00:13:03.866498 1 controller.go:198] Started VA processing "csi-f6af658de6a4535cf2ffaa4483ffbd73aa88881bb61c6e4ea870f2e0319710ab" I0618 00:13:03.866516 1 controller.go:213] Skipping VolumeAttachment csi-f6af658de6a4535cf2ffaa4483ffbd73aa88881bb61c6e4ea870f2e0319710ab for attacher csi-mock-csi-mock-volumes-4171 I0618 00:13:03.872807 1 controller.go:232] Started PV processing "pvc-2f6f4510-b794-4a10-8ca4-c3fe94803867" I0618 00:13:03.872826 1 csi_handler.go:591] CSIHandler: processing PV "pvc-2f6f4510-b794-4a10-8ca4-c3fe94803867" I0618 00:13:03.872834 1 csi_handler.go:595] CSIHandler: processing PV "pvc-2f6f4510-b794-4a10-8ca4-c3fe94803867": no deletion timestamp, ignoring I0618 00:13:14.140062 1 controller.go:198] Started VA processing "csi-35c2c9ec7d5aed383ea0a5ba5122bb4eec9a4511c579973541a45cc4b0c4b96e" I0618 00:13:14.140083 1 controller.go:213] Skipping VolumeAttachment csi-35c2c9ec7d5aed383ea0a5ba5122bb4eec9a4511c579973541a45cc4b0c4b96e for attacher csi-mock-csi-mock-volumes-4040 I0618 00:13:14.147958 1 controller.go:232] Started PV processing "pvc-6be776ff-bcea-4de0-a66d-9456ef480c9e" I0618 00:13:14.147972 1 csi_handler.go:591] CSIHandler: processing PV "pvc-6be776ff-bcea-4de0-a66d-9456ef480c9e" I0618 00:13:14.147979 1 csi_handler.go:595] CSIHandler: processing PV "pvc-6be776ff-bcea-4de0-a66d-9456ef480c9e": no deletion timestamp, ignoring I0618 00:13:14.148557 1 controller.go:198] Started VA processing "csi-d1cb48f0a673ae4b30d69335218d194fb380ad25a49c4281483d6087004537fe" I0618 00:13:14.148564 1 csi_handler.go:209] CSIHandler: processing VA "csi-d1cb48f0a673ae4b30d69335218d194fb380ad25a49c4281483d6087004537fe" I0618 00:13:14.148569 1 csi_handler.go:260] Starting detach operation for "csi-d1cb48f0a673ae4b30d69335218d194fb380ad25a49c4281483d6087004537fe" I0618 00:13:14.148596 1 csi_handler.go:267] Detaching "csi-d1cb48f0a673ae4b30d69335218d194fb380ad25a49c4281483d6087004537fe" I0618 00:13:14.148605 1 csi_handler.go:689] Found NodeID csi-mock-csi-mock-volumes-6733 in CSINode node2 I0618 00:13:14.148627 1 connection.go:182] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume I0618 00:13:14.148631 1 connection.go:183] GRPC request: {"node_id":"csi-mock-csi-mock-volumes-6733","volume_id":"4"} I0618 00:13:14.149374 1 connection.go:185] GRPC response: {} I0618 00:13:14.149671 1 connection.go:186] GRPC error: I0618 00:13:14.149679 1 csi_handler.go:549] Detached "csi-d1cb48f0a673ae4b30d69335218d194fb380ad25a49c4281483d6087004537fe" I0618 00:13:14.149691 1 util.go:76] Marking as detached "csi-d1cb48f0a673ae4b30d69335218d194fb380ad25a49c4281483d6087004537fe" I0618 00:13:14.154751 1 controller.go:232] Started PV processing "pvc-38229df3-fe87-4b3d-a1ab-65de2ca36a09" I0618 00:13:14.154761 1 csi_handler.go:591] CSIHandler: processing PV "pvc-38229df3-fe87-4b3d-a1ab-65de2ca36a09" I0618 00:13:14.154768 1 csi_handler.go:595] CSIHandler: processing PV "pvc-38229df3-fe87-4b3d-a1ab-65de2ca36a09": no deletion timestamp, ignoring I0618 00:13:14.154908 1 util.go:90] Finalizer removed from "csi-d1cb48f0a673ae4b30d69335218d194fb380ad25a49c4281483d6087004537fe" I0618 00:13:14.154917 1 csi_handler.go:280] Fully detached "csi-d1cb48f0a673ae4b30d69335218d194fb380ad25a49c4281483d6087004537fe" I0618 00:13:14.154922 1 csi_handler.go:225] CSIHandler: finished processing "csi-d1cb48f0a673ae4b30d69335218d194fb380ad25a49c4281483d6087004537fe" I0618 00:13:17.076312 1 controller.go:232] Started PV processing "pvc-38229df3-fe87-4b3d-a1ab-65de2ca36a09" I0618 00:13:17.076328 1 csi_handler.go:591] CSIHandler: processing PV "pvc-38229df3-fe87-4b3d-a1ab-65de2ca36a09" I0618 00:13:17.076341 1 csi_handler.go:633] CSIHandler: processing PV "pvc-38229df3-fe87-4b3d-a1ab-65de2ca36a09": no VA found, removing finalizer E0618 00:13:17.081315 1 csi_handler.go:650] Failed to remove finalizer from PV "pvc-38229df3-fe87-4b3d-a1ab-65de2ca36a09": PersistentVolume "pvc-38229df3-fe87-4b3d-a1ab-65de2ca36a09" is invalid: metadata.finalizers: Forbidden: no new finalizers can be added if the object is being deleted, found new finalizers []string{"kubernetes.io/pv-protection"} I0618 00:13:17.081353 1 controller.go:232] Started PV processing "pvc-38229df3-fe87-4b3d-a1ab-65de2ca36a09" I0618 00:13:17.081361 1 csi_handler.go:591] CSIHandler: processing PV "pvc-38229df3-fe87-4b3d-a1ab-65de2ca36a09" I0618 00:13:17.081374 1 csi_handler.go:633] CSIHandler: processing PV "pvc-38229df3-fe87-4b3d-a1ab-65de2ca36a09": no VA found, removing finalizer I0618 00:13:17.087040 1 csi_handler.go:655] Removed finalizer from PV "pvc-38229df3-fe87-4b3d-a1ab-65de2ca36a09" I0618 00:13:18.081604 1 controller.go:232] Started PV processing "pvc-38229df3-fe87-4b3d-a1ab-65de2ca36a09" I0618 00:13:18.081650 1 controller.go:239] PV "pvc-38229df3-fe87-4b3d-a1ab-65de2ca36a09" deleted, ignoring ==== end of pod log for container csi-mockplugin-attacher-0/csi-attacher ====