1. Turn your back to her.
Turning your back to a woman that is already attracted to you subconsciously shows her that you are used to flirting with attractive woman and you have lots of options.

2. Look over your shoulder and smile.
Although you're just being playful here, You still have to smile to show her you are still interested -- otherwise she may think you aren't interested and move on. Creating too much attraction will ruin your chances

3. Count to 4 in your head.
Resist the urge to turn around immediately. Try taking a couple of slow steps away from her while you're turned away.