In Sanjak of Zvornik until the end of 16th century most of the islamized people were vlachs (aka Serbian social caste) who arrived there from Herzegovina.
Of course your ancestors were martlolozi. They became Ottoman local elite when they converted on Islam. Majority of martolozi converted on Islam, minority stayed Orthodoxes (martolog caste disappeared until late 16th century).