Its safe to remove. But we've to be careful a bit.
go through this KB to understand .dvsdata deletion.
VMware KB: When removing a VMFS, is it safe to remove the .dvsData folder and subfolders?
this KB will give details on HA folder deletion
http://arritdor.e-wilkin.com/2012/03/removing-vmfs-datastore.html