No, a folder cannot be renamed if files are in use. To rename the folder only, you may shutdown the VM, remove it from the inventory, rename the folder and add the VM to the inventory again.
To rename the folder (as well as the files) with the VM powered on, is to rename the VM in the inventory, and then use storage vMotion to migrate the VM to another datastore.
André