VMware has an article for this error, but I think it should also mention to check templates. I had a template that was not showing it was being used by the vDS. The template was still tied to the vDS since it was using a port group on the vDS before it was converted to a template. I converted template to a VM, changed the port group, and converted the VM back to a template.  Then I was able to remove the host from the vDS.

The resource ‘XXXX’ is in use. vDS NAME port XXXX is still on host NAME connected to NAME nic=4000 type-vmNic