MST stands for Mystery Science Theatre, (also known as Mystery Science Theater 3000, or MST3K) originally a popular television programme. In the show a guy called Joel (and later, a guy called Mike) was trapped in a satellite with a bunch of talking (and sarcastic) robots - namely, Tom Servo, a small but thick red torch with a gumball head and a soothing baritone, and Crow, a crazy, semi-cosplaying robot with funny eyes and a bowling pin beak. His bosses, the 'Mads', back on Earth (both Dr. and Pearl Forrester) and their henchmen sent him cheesy B Movies and monitored him while he watched them. Joel/Mike and the 'Bots made sarcastic, usually humorous comments about the movie while they watched (sometimes referred to 'riffing'), generally as an attempt to save themselves from complete madness - though said madness has claimed Crow at least once.
In fanfiction, MST refers to an author going through a piece of fanfic and adding their own comments, as either themselves, or other characters. This is usually merciless. Originally, MSTing used the characters from the show, but since then people have branched out.