The fixture was in danger of being marred by two contentious decisions made by video assistant referees that resulted in a pair of Nabil Bentaleb spot-kicks putting Schalke ahead 2-1 at the interval.
Schalke went from having legitimate hopes of their first Champions League quarter-final berth in eight years to facing the prospect of a likely round-of-16 elimination in a matter of moments. It won't be plain sailing for City, though, with Fernandinho and Otamendi suspended for the second leg on March 12.
The English champions appeared to be cruising to another victory when David Silva preyed on some sloppy Schalke defending and teed up Sergio Aguero for an easy tap-in after 18 minutes.
However, two quick penalty calls turned the tide of the match. Following a lengthy delay, Otamendi was adjudged to have handled when he was trying to move his arm away from the ball, and Fernandinho was punished near the break when Salif Sane seemed to exaggerate slight contact from the Brazilian.
Pep Guardiola cut a frustrated figure when he responded to Otamendi's dismissal by introducing Vincent Kompany in place of David Silva, but it was the set-piece artistry from another substitution, Sane, that ignited a ferocious fightback.
Sterling latched onto a long ball from goalkeeper Ederson for the crucial finish, notching his second goal of this season's competition.