In most cases it's indeed related to north africans but I wouldn't be surprised if some portuguese have legit ssa admixture that directly came from a sub-saharan sources because of their history (I've read about "students" from the kingdom of Kongo coming studying in Portugal and if I remember correctly we're talking about thousands of individuals. There is also the case of cap verdeans)