==== start of pod log for container csi-mockplugin-attacher-0/csi-attacher ==== I0525 11:58:26.112179 1 main.go:91] Version: v2.2.0-0-g97411fa7 I0525 11:58:26.113942 1 connection.go:153] Connecting to unix:///csi/csi.sock I0525 11:58:27.121966 1 common.go:111] Probing CSI driver for readiness I0525 11:58:27.122014 1 connection.go:182] GRPC call: /csi.v1.Identity/Probe I0525 11:58:27.122025 1 connection.go:183] GRPC request: {} I0525 11:58:27.126179 1 connection.go:185] GRPC response: {"ready":{"value":true}} I0525 11:58:27.127306 1 connection.go:186] GRPC error: I0525 11:58:27.127336 1 connection.go:182] GRPC call: /csi.v1.Identity/GetPluginInfo I0525 11:58:27.127346 1 connection.go:183] GRPC request: {} I0525 11:58:27.128899 1 connection.go:185] GRPC response: {"manifest":{"url":"https://github.com/kubernetes-csi/csi-test/mock"},"name":"csi-mock-csi-mock-volumes-2982","vendor_version":"0.3.0"} I0525 11:58:27.129927 1 connection.go:186] GRPC error: I0525 11:58:27.129945 1 main.go:136] CSI driver name: "csi-mock-csi-mock-volumes-2982" W0525 11:58:27.129959 1 metrics.go:142] metrics endpoint will not be started because `metrics-address` was not specified. I0525 11:58:27.129971 1 connection.go:182] GRPC call: /csi.v1.Identity/GetPluginCapabilities I0525 11:58:27.129980 1 connection.go:183] GRPC request: {} I0525 11:58:27.131349 1 connection.go:185] GRPC response: {"capabilities":[{"Type":{"Service":{"type":1}}},{"Type":{"VolumeExpansion":{"type":1}}}]} I0525 11:58:27.135849 1 connection.go:186] GRPC error: I0525 11:58:27.135874 1 connection.go:182] GRPC call: /csi.v1.Controller/ControllerGetCapabilities I0525 11:58:27.135888 1 connection.go:183] GRPC request: {} I0525 11:58:27.137875 1 connection.go:185] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":3}}},{"Type":{"Rpc":{"type":10}}},{"Type":{"Rpc":{"type":4}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":8}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":12}}},{"Type":{"Rpc":{"type":11}}},{"Type":{"Rpc":{"type":2}}},{"Type":{"Rpc":{"type":9}}}]} I0525 11:58:27.149087 1 connection.go:186] GRPC error: I0525 11:58:27.149163 1 main.go:162] CSI driver supports ControllerPublishUnpublish, using real CSI handler I0525 11:58:27.149183 1 connection.go:182] GRPC call: /csi.v1.Controller/ControllerGetCapabilities I0525 11:58:27.149191 1 connection.go:183] GRPC request: {} I0525 11:58:27.150542 1 connection.go:185] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":3}}},{"Type":{"Rpc":{"type":10}}},{"Type":{"Rpc":{"type":4}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":8}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":12}}},{"Type":{"Rpc":{"type":11}}},{"Type":{"Rpc":{"type":2}}},{"Type":{"Rpc":{"type":9}}}]} I0525 11:58:27.159468 1 connection.go:186] GRPC error: I0525 11:58:27.159486 1 main.go:175] CSI driver supports list volumes published nodes. Using capability to reconcile volume attachment objects with actual backend state I0525 11:58:27.159611 1 controller.go:121] Starting CSI attacher I0525 11:58:27.160704 1 reflector.go:153] Starting reflector *v1.PersistentVolume (10m0s) from k8s.io/client-go/informers/factory.go:135 I0525 11:58:27.160741 1 reflector.go:188] Listing and watching *v1.PersistentVolume from k8s.io/client-go/informers/factory.go:135 I0525 11:58:27.160751 1 reflector.go:153] Starting reflector *v1beta1.CSINode (10m0s) from k8s.io/client-go/informers/factory.go:135 I0525 11:58:27.160778 1 reflector.go:188] Listing and watching *v1beta1.CSINode from k8s.io/client-go/informers/factory.go:135 I0525 11:58:27.160754 1 reflector.go:153] Starting reflector *v1beta1.VolumeAttachment (10m0s) from k8s.io/client-go/informers/factory.go:135 I0525 11:58:27.160881 1 reflector.go:188] Listing and watching *v1beta1.VolumeAttachment from k8s.io/client-go/informers/factory.go:135 I0525 11:58:27.259885 1 shared_informer.go:227] caches populated I0525 11:58:27.259958 1 controller.go:198] Started VA processing "csi-a9943b9f361ed62c6fbf90d86ae96df60c0c67325c0009bf9a55c27085554302" I0525 11:58:27.259978 1 controller.go:213] Skipping VolumeAttachment csi-a9943b9f361ed62c6fbf90d86ae96df60c0c67325c0009bf9a55c27085554302 for attacher csi-mock-csi-mock-volumes-4911 I0525 11:58:27.259991 1 csi_handler.go:116] Reconciling VolumeAttachments with driver backend state I0525 11:58:27.259996 1 controller.go:198] Started VA processing "csi-9be1a89999d615993da7da4bb4ae63e1453e5cfb34c514e44f140a682958ed5d" I0525 11:58:27.260007 1 controller.go:213] Skipping VolumeAttachment csi-9be1a89999d615993da7da4bb4ae63e1453e5cfb34c514e44f140a682958ed5d for attacher csi-mock-csi-mock-volumes-6581 I0525 11:58:27.260015 1 controller.go:198] Started VA processing "csi-f866cc7cad364f671f57deb7c32cddd3d86a84506df2a8df0d066b64bee70e12" I0525 11:58:27.260021 1 controller.go:213] Skipping VolumeAttachment csi-f866cc7cad364f671f57deb7c32cddd3d86a84506df2a8df0d066b64bee70e12 for attacher csi-mock-csi-mock-volumes-6755 I0525 11:58:27.260024 1 connection.go:182] GRPC call: /csi.v1.Controller/ListVolumes I0525 11:58:27.260034 1 connection.go:183] GRPC request: {} I0525 11:58:27.262766 1 connection.go:185] GRPC response: {"entries":[{"status":{"published_node_ids":["csi-mock-csi-mock-volumes-2982"]},"volume":{"capacity_bytes":107374182400,"volume_context":{"name":"Mock Volume 1"},"volume_id":"1"}},{"status":{"published_node_ids":["csi-mock-csi-mock-volumes-2982"]},"volume":{"capacity_bytes":107374182400,"volume_context":{"name":"Mock Volume 2"},"volume_id":"2"}},{"status":{"published_node_ids":["csi-mock-csi-mock-volumes-2982"]},"volume":{"capacity_bytes":107374182400,"volume_context":{"name":"Mock Volume 3"},"volume_id":"3"}}]} I0525 11:58:27.272220 1 connection.go:186] GRPC error: W0525 11:58:27.272248 1 csi_handler.go:159] Failed to repair volume handle for driver csi-mock-csi-mock-volumes-4911: could not find In-Tree driver name for CSI plugin csi-mock-csi-mock-volumes-4911 W0525 11:58:27.272260 1 csi_handler.go:159] Failed to repair volume handle for driver csi-mock-csi-mock-volumes-6581: could not find In-Tree driver name for CSI plugin csi-mock-csi-mock-volumes-6581 W0525 11:58:27.272267 1 csi_handler.go:159] Failed to repair volume handle for driver csi-mock-csi-mock-volumes-6755: could not find In-Tree driver name for CSI plugin csi-mock-csi-mock-volumes-6755 I0525 11:58:32.598024 1 controller.go:198] Started VA processing "csi-a9943b9f361ed62c6fbf90d86ae96df60c0c67325c0009bf9a55c27085554302" I0525 11:58:32.598063 1 controller.go:213] Skipping VolumeAttachment csi-a9943b9f361ed62c6fbf90d86ae96df60c0c67325c0009bf9a55c27085554302 for attacher csi-mock-csi-mock-volumes-4911 I0525 11:58:32.606928 1 controller.go:232] Started PV processing "pvc-26e64179-c99e-4ee0-ba37-51ac9288c0bf" I0525 11:58:32.606967 1 csi_handler.go:591] CSIHandler: processing PV "pvc-26e64179-c99e-4ee0-ba37-51ac9288c0bf" I0525 11:58:32.606995 1 csi_handler.go:595] CSIHandler: processing PV "pvc-26e64179-c99e-4ee0-ba37-51ac9288c0bf": no deletion timestamp, ignoring I0525 11:58:36.813481 1 controller.go:198] Started VA processing "csi-3136da6543b3febae852b37caad09dde07a8c2121633bb3930c6e186619e254c" I0525 11:58:36.813558 1 csi_handler.go:209] CSIHandler: processing VA "csi-3136da6543b3febae852b37caad09dde07a8c2121633bb3930c6e186619e254c" I0525 11:58:36.813580 1 csi_handler.go:236] Attaching "csi-3136da6543b3febae852b37caad09dde07a8c2121633bb3930c6e186619e254c" I0525 11:58:36.813602 1 csi_handler.go:396] Starting attach operation for "csi-3136da6543b3febae852b37caad09dde07a8c2121633bb3930c6e186619e254c" I0525 11:58:36.813732 1 csi_handler.go:335] Adding finalizer to PV "pvc-9bbe0f07-a69c-4365-a7a5-b47244a95bfa" I0525 11:58:36.824840 1 csi_handler.go:344] PV finalizer added to "pvc-9bbe0f07-a69c-4365-a7a5-b47244a95bfa" I0525 11:58:36.824921 1 csi_handler.go:689] Found NodeID csi-mock-csi-mock-volumes-2982 in CSINode v1.21-worker2 I0525 11:58:36.824985 1 csi_handler.go:297] VA finalizer added to "csi-3136da6543b3febae852b37caad09dde07a8c2121633bb3930c6e186619e254c" I0525 11:58:36.825980 1 csi_handler.go:311] NodeID annotation added to "csi-3136da6543b3febae852b37caad09dde07a8c2121633bb3930c6e186619e254c" I0525 11:58:36.832401 1 connection.go:182] GRPC call: /csi.v1.Controller/ControllerPublishVolume I0525 11:58:36.832452 1 connection.go:183] GRPC request: {"node_id":"csi-mock-csi-mock-volumes-2982","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":1}},"volume_context":{"name":"pvc-9bbe0f07-a69c-4365-a7a5-b47244a95bfa","storage.kubernetes.io/csiProvisionerIdentity":"1621943907135-8081-csi-mock-csi-mock-volumes-2982"},"volume_id":"4"} I0525 11:58:36.836891 1 connection.go:185] GRPC response: {"publish_context":{"device":"/dev/mock","readonly":"false"}} I0525 11:58:36.838111 1 connection.go:186] GRPC error: I0525 11:58:36.838128 1 csi_handler.go:249] Attached "csi-3136da6543b3febae852b37caad09dde07a8c2121633bb3930c6e186619e254c" I0525 11:58:36.838142 1 util.go:35] Marking as attached "csi-3136da6543b3febae852b37caad09dde07a8c2121633bb3930c6e186619e254c" I0525 11:58:36.843797 1 util.go:48] Marked as attached "csi-3136da6543b3febae852b37caad09dde07a8c2121633bb3930c6e186619e254c" I0525 11:58:36.843839 1 csi_handler.go:255] Fully attached "csi-3136da6543b3febae852b37caad09dde07a8c2121633bb3930c6e186619e254c" I0525 11:58:36.843854 1 csi_handler.go:225] CSIHandler: finished processing "csi-3136da6543b3febae852b37caad09dde07a8c2121633bb3930c6e186619e254c" I0525 11:58:36.843897 1 controller.go:198] Started VA processing "csi-3136da6543b3febae852b37caad09dde07a8c2121633bb3930c6e186619e254c" I0525 11:58:36.843911 1 csi_handler.go:209] CSIHandler: processing VA "csi-3136da6543b3febae852b37caad09dde07a8c2121633bb3930c6e186619e254c" I0525 11:58:36.843926 1 csi_handler.go:231] "csi-3136da6543b3febae852b37caad09dde07a8c2121633bb3930c6e186619e254c" is already attached I0525 11:58:36.843939 1 csi_handler.go:225] CSIHandler: finished processing "csi-3136da6543b3febae852b37caad09dde07a8c2121633bb3930c6e186619e254c" I0525 11:59:02.895679 1 controller.go:198] Started VA processing "csi-3136da6543b3febae852b37caad09dde07a8c2121633bb3930c6e186619e254c" I0525 11:59:02.895724 1 csi_handler.go:209] CSIHandler: processing VA "csi-3136da6543b3febae852b37caad09dde07a8c2121633bb3930c6e186619e254c" I0525 11:59:02.895736 1 csi_handler.go:260] Starting detach operation for "csi-3136da6543b3febae852b37caad09dde07a8c2121633bb3930c6e186619e254c" I0525 11:59:02.895775 1 csi_handler.go:267] Detaching "csi-3136da6543b3febae852b37caad09dde07a8c2121633bb3930c6e186619e254c" I0525 11:59:02.895868 1 csi_handler.go:689] Found NodeID csi-mock-csi-mock-volumes-2982 in CSINode v1.21-worker2 I0525 11:59:02.896012 1 connection.go:182] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume I0525 11:59:02.896053 1 connection.go:183] GRPC request: {"node_id":"csi-mock-csi-mock-volumes-2982","volume_id":"4"} I0525 11:59:02.898900 1 connection.go:185] GRPC response: {} I0525 11:59:02.899891 1 connection.go:186] GRPC error: I0525 11:59:02.899903 1 csi_handler.go:549] Detached "csi-3136da6543b3febae852b37caad09dde07a8c2121633bb3930c6e186619e254c" I0525 11:59:02.899939 1 util.go:76] Marking as detached "csi-3136da6543b3febae852b37caad09dde07a8c2121633bb3930c6e186619e254c" I0525 11:59:02.909733 1 controller.go:232] Started PV processing "pvc-9bbe0f07-a69c-4365-a7a5-b47244a95bfa" I0525 11:59:02.909771 1 csi_handler.go:591] CSIHandler: processing PV "pvc-9bbe0f07-a69c-4365-a7a5-b47244a95bfa" I0525 11:59:02.909797 1 csi_handler.go:595] CSIHandler: processing PV "pvc-9bbe0f07-a69c-4365-a7a5-b47244a95bfa": no deletion timestamp, ignoring I0525 11:59:02.909959 1 util.go:90] Finalizer removed from "csi-3136da6543b3febae852b37caad09dde07a8c2121633bb3930c6e186619e254c" I0525 11:59:02.909987 1 csi_handler.go:280] Fully detached "csi-3136da6543b3febae852b37caad09dde07a8c2121633bb3930c6e186619e254c" I0525 11:59:02.910002 1 csi_handler.go:225] CSIHandler: finished processing "csi-3136da6543b3febae852b37caad09dde07a8c2121633bb3930c6e186619e254c" I0525 11:59:03.787843 1 controller.go:232] Started PV processing "pvc-9bbe0f07-a69c-4365-a7a5-b47244a95bfa" I0525 11:59:03.787887 1 csi_handler.go:591] CSIHandler: processing PV "pvc-9bbe0f07-a69c-4365-a7a5-b47244a95bfa" I0525 11:59:03.787934 1 csi_handler.go:633] CSIHandler: processing PV "pvc-9bbe0f07-a69c-4365-a7a5-b47244a95bfa": no VA found, removing finalizer E0525 11:59:03.798137 1 csi_handler.go:650] Failed to remove finalizer from PV "pvc-9bbe0f07-a69c-4365-a7a5-b47244a95bfa": PersistentVolume "pvc-9bbe0f07-a69c-4365-a7a5-b47244a95bfa" is invalid: metadata.finalizers: Forbidden: no new finalizers can be added if the object is being deleted, found new finalizers []string{"kubernetes.io/pv-protection"} I0525 11:59:03.798248 1 controller.go:232] Started PV processing "pvc-9bbe0f07-a69c-4365-a7a5-b47244a95bfa" I0525 11:59:03.798297 1 csi_handler.go:591] CSIHandler: processing PV "pvc-9bbe0f07-a69c-4365-a7a5-b47244a95bfa" I0525 11:59:03.798339 1 csi_handler.go:633] CSIHandler: processing PV "pvc-9bbe0f07-a69c-4365-a7a5-b47244a95bfa": no VA found, removing finalizer I0525 11:59:03.807590 1 csi_handler.go:655] Removed finalizer from PV "pvc-9bbe0f07-a69c-4365-a7a5-b47244a95bfa" I0525 11:59:04.798443 1 controller.go:232] Started PV processing "pvc-9bbe0f07-a69c-4365-a7a5-b47244a95bfa" I0525 11:59:04.798500 1 controller.go:239] PV "pvc-9bbe0f07-a69c-4365-a7a5-b47244a95bfa" deleted, ignoring I0525 11:59:08.720654 1 controller.go:198] Started VA processing "csi-d7b588a91c3ea00299e475f306105561217058b15b02c0dcbb8552f05f5e0847" I0525 11:59:08.720738 1 controller.go:213] Skipping VolumeAttachment csi-d7b588a91c3ea00299e475f306105561217058b15b02c0dcbb8552f05f5e0847 for attacher csi-mock-csi-mock-volumes-844 I0525 11:59:08.738088 1 controller.go:198] Started VA processing "csi-d7b588a91c3ea00299e475f306105561217058b15b02c0dcbb8552f05f5e0847" I0525 11:59:08.738119 1 controller.go:213] Skipping VolumeAttachment csi-d7b588a91c3ea00299e475f306105561217058b15b02c0dcbb8552f05f5e0847 for attacher csi-mock-csi-mock-volumes-844 I0525 11:59:08.750190 1 controller.go:198] Started VA processing "csi-d7b588a91c3ea00299e475f306105561217058b15b02c0dcbb8552f05f5e0847" I0525 11:59:08.750257 1 controller.go:213] Skipping VolumeAttachment csi-d7b588a91c3ea00299e475f306105561217058b15b02c0dcbb8552f05f5e0847 for attacher csi-mock-csi-mock-volumes-844 ==== end of pod log for container csi-mockplugin-attacher-0/csi-attacher ====