==== start of pod log for container csi-mockplugin-0/csi-provisioner ==== W0418 18:16:18.904222 1 feature_gate.go:237] Setting GA feature gate Topology=true. It will be removed in a future release. I0418 18:16:18.904273 1 feature_gate.go:245] feature gates: &{map[Topology:true]} I0418 18:16:18.904303 1 csi-provisioner.go:150] Version: v3.3.0 I0418 18:16:18.904312 1 csi-provisioner.go:173] Building kube configs for running in cluster... I0418 18:16:18.904882 1 connection.go:154] Connecting to unix:///csi/csi.sock I0418 18:16:19.907894 1 common.go:111] Probing CSI driver for readiness I0418 18:16:19.907952 1 connection.go:183] GRPC call: /csi.v1.Identity/Probe I0418 18:16:19.907965 1 connection.go:184] GRPC request: {} I0418 18:16:19.920578 1 connection.go:186] GRPC response: {} I0418 18:16:19.920701 1 connection.go:187] GRPC error: I0418 18:16:19.920722 1 connection.go:183] GRPC call: /csi.v1.Identity/GetPluginInfo I0418 18:16:19.920731 1 connection.go:184] GRPC request: {} I0418 18:16:19.921829 1 connection.go:186] GRPC response: {"name":"csi-mock-csi-mock-volumes-3590","vendor_version":"v1.9.0"} I0418 18:16:19.921949 1 connection.go:187] GRPC error: I0418 18:16:19.921973 1 csi-provisioner.go:217] Detected CSI driver csi-mock-csi-mock-volumes-3590 I0418 18:16:19.921990 1 connection.go:183] GRPC call: /csi.v1.Identity/GetPluginCapabilities I0418 18:16:19.921999 1 connection.go:184] GRPC request: {} I0418 18:16:19.923077 1 connection.go:186] GRPC response: {"capabilities":[{"Type":{"Service":{"type":1}}},{"Type":{"Service":{"type":2}}}]} I0418 18:16:19.923217 1 connection.go:187] GRPC error: I0418 18:16:19.923231 1 connection.go:183] GRPC call: /csi.v1.Controller/ControllerGetCapabilities I0418 18:16:19.923236 1 connection.go:184] GRPC request: {} I0418 18:16:19.924415 1 connection.go:186] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":12}}},{"Type":{"Rpc":{"type":4}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":3}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":11}}},{"Type":{"Rpc":{"type":13}}},{"Type":{"Rpc":{"type":9}}},{"Type":{"Rpc":{"type":2}}}]} I0418 18:16:19.924737 1 connection.go:187] GRPC error: I0418 18:16:19.924849 1 csi-provisioner.go:286] CSI driver supports PUBLISH_UNPUBLISH_VOLUME, watching VolumeAttachments I0418 18:16:19.925831 1 controller.go:732] Using saving PVs to API server in background I0418 18:16:19.926410 1 reflector.go:221] Starting reflector *v1.CSINode (1h0m0s) from k8s.io/client-go/informers/factory.go:134 I0418 18:16:19.926425 1 reflector.go:221] Starting reflector *v1.VolumeAttachment (1h0m0s) from k8s.io/client-go/informers/factory.go:134 I0418 18:16:19.926451 1 reflector.go:257] Listing and watching *v1.CSINode from k8s.io/client-go/informers/factory.go:134 I0418 18:16:19.926457 1 reflector.go:257] Listing and watching *v1.VolumeAttachment from k8s.io/client-go/informers/factory.go:134 I0418 18:16:19.926477 1 reflector.go:221] Starting reflector *v1.Node (1h0m0s) from k8s.io/client-go/informers/factory.go:134 I0418 18:16:19.926501 1 reflector.go:257] Listing and watching *v1.Node from k8s.io/client-go/informers/factory.go:134 I0418 18:16:19.926478 1 reflector.go:221] Starting reflector *v1.StorageClass (1h0m0s) from k8s.io/client-go/informers/factory.go:134 I0418 18:16:19.926528 1 reflector.go:257] Listing and watching *v1.StorageClass from k8s.io/client-go/informers/factory.go:134 I0418 18:16:19.926602 1 reflector.go:221] Starting reflector *v1.PersistentVolumeClaim (15m0s) from k8s.io/client-go/informers/factory.go:134 I0418 18:16:19.926633 1 reflector.go:257] Listing and watching *v1.PersistentVolumeClaim from k8s.io/client-go/informers/factory.go:134 I0418 18:16:20.026281 1 shared_informer.go:285] caches populated I0418 18:16:20.026340 1 shared_informer.go:285] caches populated I0418 18:16:20.026351 1 shared_informer.go:285] caches populated I0418 18:16:20.026362 1 shared_informer.go:285] caches populated I0418 18:16:20.026371 1 shared_informer.go:285] caches populated I0418 18:16:20.026392 1 controller.go:811] Starting provisioner controller csi-mock-csi-mock-volumes-3590_csi-mockplugin-0_9f01e0a8-c892-4ccf-b1d6-86c0b546abbe! I0418 18:16:20.026438 1 clone_controller.go:66] Starting CloningProtection controller I0418 18:16:20.026522 1 clone_controller.go:82] Started CloningProtection controller I0418 18:16:20.026537 1 volume_store.go:97] Starting save volume queue I0418 18:16:20.026732 1 reflector.go:221] Starting reflector *v1.PersistentVolume (15m0s) from sigs.k8s.io/sig-storage-lib-external-provisioner/v8/controller/controller.go:845 I0418 18:16:20.026753 1 reflector.go:257] Listing and watching *v1.PersistentVolume from sigs.k8s.io/sig-storage-lib-external-provisioner/v8/controller/controller.go:845 I0418 18:16:20.026753 1 reflector.go:221] Starting reflector *v1.StorageClass (15m0s) from sigs.k8s.io/sig-storage-lib-external-provisioner/v8/controller/controller.go:848 I0418 18:16:20.026775 1 reflector.go:257] Listing and watching *v1.StorageClass from sigs.k8s.io/sig-storage-lib-external-provisioner/v8/controller/controller.go:848 I0418 18:16:20.127444 1 shared_informer.go:285] caches populated I0418 18:16:20.127521 1 controller.go:860] Started provisioner controller csi-mock-csi-mock-volumes-3590_csi-mockplugin-0_9f01e0a8-c892-4ccf-b1d6-86c0b546abbe! I0418 18:16:27.272798 1 controller.go:1337] provision "csi-mock-volumes-3590/pvc-bws7x" class "csi-mock-volumes-3590-sc6f88m": started I0418 18:16:27.273114 1 event.go:285] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"csi-mock-volumes-3590", Name:"pvc-bws7x", UID:"e66e3187-306f-4ce2-a9a8-218dd2fd2c61", APIVersion:"v1", ResourceVersion:"95909", FieldPath:""}): type: 'Normal' reason: 'Provisioning' External provisioner is provisioning volume for claim "csi-mock-volumes-3590/pvc-bws7x" I0418 18:16:27.273205 1 connection.go:183] GRPC call: /csi.v1.Controller/CreateVolume I0418 18:16:27.273231 1 connection.go:184] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.hostpath.csi/node":"v126-worker"}},{"segments":{"topology.hostpath.csi/node":"v126-worker2"}}],"requisite":[{"segments":{"topology.hostpath.csi/node":"v126-worker"}},{"segments":{"topology.hostpath.csi/node":"v126-worker2"}}]},"capacity_range":{"required_bytes":1073741824},"name":"pvc-e66e3187-306f-4ce2-a9a8-218dd2fd2c61","volume_capabilities":[{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}}]} I0418 18:16:27.278255 1 connection.go:186] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.hostpath.csi/node":"v126-worker"}}],"capacity_bytes":1073741824,"volume_id":"c5af3855-fdaf-11ee-8161-7aee35f14ad8"}} I0418 18:16:27.278438 1 connection.go:187] GRPC error: I0418 18:16:27.278452 1 controller.go:812] create volume rep: {CapacityBytes:1073741824 VolumeId:c5af3855-fdaf-11ee-8161-7aee35f14ad8 VolumeContext:map[] ContentSource: AccessibleTopology:[segments: ] XXX_NoUnkeyedLiteral:{} XXX_unrecognized:[] XXX_sizecache:0} I0418 18:16:27.278540 1 controller.go:881] createVolumeOperation: set annotation [volume.kubernetes.io/provisioner-deletion-secret-namespace/volume.kubernetes.io/provisioner-deletion-secret-name] on pv [pvc-e66e3187-306f-4ce2-a9a8-218dd2fd2c61]. I0418 18:16:27.278579 1 controller.go:906] successfully created PV pvc-e66e3187-306f-4ce2-a9a8-218dd2fd2c61 for PVC pvc-bws7x and csi volume name c5af3855-fdaf-11ee-8161-7aee35f14ad8 I0418 18:16:27.278601 1 controller.go:922] successfully created PV {GCEPersistentDisk:nil AWSElasticBlockStore:nil HostPath:nil Glusterfs:nil NFS:nil RBD:nil ISCSI:nil Cinder:nil CephFS:nil FC:nil Flocker:nil FlexVolume:nil AzureFile:nil VsphereVolume:nil Quobyte:nil AzureDisk:nil PhotonPersistentDisk:nil PortworxVolume:nil ScaleIO:nil Local:nil StorageOS:nil CSI:&CSIPersistentVolumeSource{Driver:csi-mock-csi-mock-volumes-3590,VolumeHandle:c5af3855-fdaf-11ee-8161-7aee35f14ad8,ReadOnly:false,FSType:ext4,VolumeAttributes:map[string]string{storage.kubernetes.io/csiProvisionerIdentity: 1713464179924-8081-csi-mock-csi-mock-volumes-3590,},ControllerPublishSecretRef:nil,NodeStageSecretRef:nil,NodePublishSecretRef:nil,ControllerExpandSecretRef:nil,NodeExpandSecretRef:nil,}} I0418 18:16:27.278845 1 controller.go:1442] provision "csi-mock-volumes-3590/pvc-bws7x" class "csi-mock-volumes-3590-sc6f88m": volume "pvc-e66e3187-306f-4ce2-a9a8-218dd2fd2c61" provisioned I0418 18:16:27.278872 1 controller.go:1455] provision "csi-mock-volumes-3590/pvc-bws7x" class "csi-mock-volumes-3590-sc6f88m": succeeded I0418 18:16:27.278883 1 volume_store.go:154] Saving volume pvc-e66e3187-306f-4ce2-a9a8-218dd2fd2c61 I0418 18:16:27.285013 1 volume_store.go:157] Volume pvc-e66e3187-306f-4ce2-a9a8-218dd2fd2c61 saved I0418 18:16:27.285097 1 controller.go:1069] Claim processing succeeded, removing PVC e66e3187-306f-4ce2-a9a8-218dd2fd2c61 from claims in progress I0418 18:16:27.285199 1 event.go:285] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"csi-mock-volumes-3590", Name:"pvc-bws7x", UID:"e66e3187-306f-4ce2-a9a8-218dd2fd2c61", APIVersion:"v1", ResourceVersion:"95909", FieldPath:""}): type: 'Normal' reason: 'ProvisioningSucceeded' Successfully provisioned volume pvc-e66e3187-306f-4ce2-a9a8-218dd2fd2c61 I0418 18:16:41.339734 1 controller.go:1471] delete "pvc-e66e3187-306f-4ce2-a9a8-218dd2fd2c61": started I0418 18:16:41.339835 1 controller.go:1261] volume pvc-e66e3187-306f-4ce2-a9a8-218dd2fd2c61 does not need any deletion secrets E0418 18:16:41.339882 1 controller.go:1481] delete "pvc-e66e3187-306f-4ce2-a9a8-218dd2fd2c61": volume deletion failed: persistentvolume pvc-e66e3187-306f-4ce2-a9a8-218dd2fd2c61 is still attached to node v126-worker W0418 18:16:41.339937 1 controller.go:989] Retrying syncing volume "pvc-e66e3187-306f-4ce2-a9a8-218dd2fd2c61", failure 0 E0418 18:16:41.339973 1 controller.go:1007] error syncing volume "pvc-e66e3187-306f-4ce2-a9a8-218dd2fd2c61": persistentvolume pvc-e66e3187-306f-4ce2-a9a8-218dd2fd2c61 is still attached to node v126-worker I0418 18:16:41.339999 1 event.go:285] Event(v1.ObjectReference{Kind:"PersistentVolume", Namespace:"", Name:"pvc-e66e3187-306f-4ce2-a9a8-218dd2fd2c61", UID:"a05813d2-0cb9-4424-8a6d-bc4f19bfd8eb", APIVersion:"v1", ResourceVersion:"96097", FieldPath:""}): type: 'Warning' reason: 'VolumeFailedDelete' persistentvolume pvc-e66e3187-306f-4ce2-a9a8-218dd2fd2c61 is still attached to node v126-worker I0418 18:16:42.340758 1 controller.go:1471] delete "pvc-e66e3187-306f-4ce2-a9a8-218dd2fd2c61": started I0418 18:16:42.340845 1 controller.go:1261] volume pvc-e66e3187-306f-4ce2-a9a8-218dd2fd2c61 does not need any deletion secrets I0418 18:16:42.340882 1 connection.go:183] GRPC call: /csi.v1.Controller/DeleteVolume I0418 18:16:42.340898 1 connection.go:184] GRPC request: {"volume_id":"c5af3855-fdaf-11ee-8161-7aee35f14ad8"} I0418 18:16:42.342952 1 connection.go:186] GRPC response: {} I0418 18:16:42.343031 1 connection.go:187] GRPC error: I0418 18:16:42.343055 1 controller.go:1486] delete "pvc-e66e3187-306f-4ce2-a9a8-218dd2fd2c61": volume deleted I0418 18:16:42.348466 1 controller.go:1531] delete "pvc-e66e3187-306f-4ce2-a9a8-218dd2fd2c61": persistentvolume deleted I0418 18:16:42.348514 1 controller.go:1536] delete "pvc-e66e3187-306f-4ce2-a9a8-218dd2fd2c61": succeeded ==== end of pod log for container csi-mockplugin-0/csi-provisioner ====