Clustering: RPC failing in clustering data store integration tests

I'm running /suites/clustering/datastore/020_crud_on_leader.txt.

The Keywords that are failing are:

Purchase cars on Leader
Purchase cars on Follower1
Purchase cars on Follower2

What's happening is that the RPCs to "buy a car" only work when executed against one of the three controllers. Strangely it's note even the car-people shard leader.

Tom Pantelis is running these test in Brocade's lab on Helium and they are passing. This bug is unique to Lithium.

Procedure:

Pull the latest stable Lithium Release. I pulled distribution-karaf-0.3.0-20150414.220637-1140.zip

Use the python cluster deploy script.

Let the test run. The shards configured themselves in this way:

car shard

{'member1': 'Follower', 'member2': 'Leader', 'member3': 'Follower'}

people shard

{'member1': 'Follower', 'member2': 'Follower', 'member3': 'Leader'}

car-people shard

{'member1': 'Leader', 'member2': 'Follower', 'member3': 'Follower'}

After the test completes, use ReST RPCs to purchase a car on member1, member2, and member2.

URL:
http://member1:8181/restconf/operations/car-purchase:buy-car

JSON:
{ "input" :

{ "car-purchase:person" : "/people:people/people:person[people:id='user9']", "car-purchase:person-id" : "user9", "car-purchase:car-id" : "9" }

}

For the first two members, there will be a 500 error in Postman, and nothing in the karaf log (there will be a separate bug addressing the log and error messages). Member3 gets a nice “200 OK” in Postman and I see an acknowledgement of the car purchase in the karaf log.

RPCs are not working on two out of the three controllers.