Actually looking at the Find Replace code there is this snippet:
Code:
// Set F&R Buttons to text-only if requested
bool frButtonsTextOnly = settings.value("frbuttonstextonly", false).toBool();
if (frButtonsTextOnly) {
SetFRButtonsTextOnly();
}
So in the Sigil ini settings there is a setting that tells Sigil to strip the icon from the FindReplace buttons. It is called "frbuttonstextonly".
There is no interface to change it but editing the Sigil v6 ini file and changing that key's value to return true should do what you want.