Switch connection keeps flapping after multiple controller restarts in a cluster

On performing multiple reboots of one or more controllers in a cluster, switch connection goes to a weird state, where it tries to send the port/group/meter feature request to the switch and when it does not get the response back, it end up disconnecting the switch and switch ownership move to the other controller. Other controller also does the same, and it does not get the response and disconnect the switch and the ownership keep changing. All the device connection to controllers keep flapping and it happens for all the devices
 
If the devices are disconnected and connected back, everything came back to the normal.