When you manipulate items in windows explorer from AWS s3 bucket , it creates folders which then get uploaded to AWS as 0 byte objects. 

You can view these in powershell by using 

Get-S3Object -BucketName “XXXXXX” -keyprefix “documents/XXXXX/XXXXX/” | where-object {$_.Size -eq 0 -and $_.Key -like “XXXX/XXXX/XXXX/12*”}

To remove ( You can only items 1000 at a time ) you can use the below

Remove-S3Object -bucketname “XXXXXX” -KeyCollection (Get-S3Object -BucketName “XXXXXX” -keyprefix “documents/XXXXX/XXXXX/” | where-object {$_.Size -eq 0 -and $_.Key -like “XXXX/XXXX/XXXX/12*”}| select -ExpandProperty Key)