What is contradictory here? Serbia like every country changed borders a lot. At one point of history it owned Zahumlje.
Origin of population is different story. Either Zahumljani were not Serbs, or they were Serbs ruled by a foreigner which I don't find likely in such tribal Slavic times.
Territories of both Serbia and Croatia had other Slavic tribes - Braničevci and Timočani in Serbia, Guduscani and Narentines in Croatia for example.