Setup:
3 control node
2 compute node
3 controller in cluster

Steps to reproduce issue:

openstack network create network_1 --provider-network-type vxlan
openstack network create network_2 --provider-network-type vxlan
openstack subnet create --network network_1 --subnet-range 30.0.0.0/24 l2_subnet_1
openstack subnet create --network network_2 --subnet-range 40.0.0.0/24 l2_subnet_2
openstack server create --image cirros --flavor cirros --nic net-id=network_1 VM1 --security-group default
openstack server create --image cirros --flavor cirros --nic net-id=network_1 VM2 --security-group default
openstack router create router1
openstack router add subnet router1 l2_subnet_1
openstack router add subnet router1 l2_subnet_2
openstack router set router1 --route destination=30.0.0.0/24,gateway=30.0.0.2
Check ping from VM2 to VM1 (expected to work, but ping fails)

Similar Issue occurs when try to ping external network:

openstack network create network_1 --provider-network-type vxlan
openstack subnet create --network network_1 --subnet-range 30.0.0.0/24 l2_subnet_1
openstack network create external-net --provider-network-type flat --provider-physical-network flat1
openstack network set external-net --external
openstack subnet create --network external-net --subnet-range 101.0.0.0/24 external-subnet --gateway 101.0.0.250 --allocation-pool start=101.0.0.4,end=101.0.0.249
openstack server create --image cirros --flavor cirros --nic net-id=network_1 VM2 --security-group default
openstack router create router2
openstack router add subnet router2 l2_subnet_1
openstack router set router2 --external-gateway external-net
openstack router set router2 --route destination=101.0.0.0/24,gateway=101.0.0.2
Check ping from VM1 to external network (expected to work, but ping fails)

Note-
1. Occurs ocassionally.
2. Add icmp rules to default security group.