Hard to say without doing performance tests. From what you said, I'd take a closer look at the outgoing network traffic (VM -> local system) and also check the local systems's disk performance as well as processes like virus scan software (on access scan).
André