Yeah, it's pretty complicated. About the only way I can make it make sense is to assume the criminals are one of the pairs of appraisers who wanted to steal one of Agasa's items (the rug). The problem with just going in, stunning Agasa, and taking it is that suspicion will naturally first fall on the appraisers who visited Agasa. Therefore, the pair had to come up with some other plan to try to steal the rug without Agasa realizing it, and make it look like random, unknown people were the culprits to avoid placing suspicion on themselves. Not wanting to actually kill anyone, they came up with this strange plan involving a desperate lost cat search in order to take the genuine rug and plant a fake rug so nobody will realize that the goal all along was the rug. If they just took the rug and Ayumi, then dumped Ayumi without the rug, it would be easier to realize that the cat thing was just some excuse they made up.Regarding which pair are the culprits, I think the middle aged couple are the baddies.
If Agasa is going to talk to anyone about his schedule, kids, and stuff, this topic would be most natural for a middle aged couple. The young girls might find it weird and the father son team made Agasa angry so pleasantries are unlikely.
In terms of their investigation methods, the middle aged couple not touching the pot is weird because makers marks are often placed on the bottom. The girl pair didn't seem very professional, like they didn't know what they were doing. The father son team did seem to know what they were doing though they didn't bring a magnifying glass, but they didn't accept Agasa's offer to buy the rug and the vase at 50000 yen (~$615 USD). I don't think the culprits would bother to stage a kidnapping for a rug worth less than that.
Finally the middle aged couple also suggested that Agasa take the vase to a real appraiser which would get him out of the house, but also not show it to anyone else, perhaps because they are worried someone might mention the rug. The way the father son team reacted to Agasa's offer, I bet they knew the value of the carpet, but they walked out on it. They also joked about how "people without an eye for these things are sad", which would most certainly goad Agasa into trying harder to get a good evaluation, not the reaction thieves would want. The girl duo didn't mention the rug at all.