User blog comment:DaNASCAT/Technical Update: August 21, 2012/@comment-3147293-20120821164626/@comment-957747-20120821184128

If you have the RTE turned off (IE: source mode), you won't see them. They should only show for the visual editor.