Booting a VM with direct switchdev port fails if the same port (VF) was used before.

How to reproduce:

using host that has 1 VF.

• create  direct switchdev port
• boot a VM with the port
• delete the VM 
• delete the port
• create  direct switchdev port
• boot a VM with switchdev port

then, the VM will timeout at spawning state.