Using HTML in Submit Button Text

I'm trying to add a span tag with triangle character into the Submit button text. The span tag gets treated as part of the text though and isn't rendered as HTML.

The value which I'm trying to use is:
Submit<span> ►</span>

For now I'm using:
Submit ►

But I want to be able to stylize the triangle with CSS to be a smaller font-size though.

Also I tried content::after but can't seem to get that to show the triangle character.

Thnaks,
DAN

Thanks Rob - works great!