The Dublin side had trailed 8-0 at the break with Tyrone Howe scoring the only try just before the interval after Adam Larkin had opened the scoring with a penalty.
Contepomi had cut the lead to two points with little time left on the clock and the northerners looked set for a surprise win before Contepomi knocked over the winner after Rowan Frost sin-binned for killing the ball in the final minute.
However Ulster were the only try scorers with not even Brian O’Driscoll – who was yellow-carded with Ulster’s Tyrone Howe on 42 minutes – could breach the Ulster try line, on a night when Howe’s opportunist score in first-half injury time proved the highlight.
The 33-year-old displayed some nifty dribbling as, having intercepted Gordon D’Arcy’s midfield pass, he managed to hack on three times to evade the retreating O’Driscoll and touch down.
Ulster fly-half Adam Larkin – deputising for the injured David Humphreys – had kicked the visitors into a third-minute lead, but a series of needless penalties saw Mark McCall’s side camped in their own half.
An off-colour Leinster fluffed well-set chances through the returning D’Arcy and Aidan McCullen.
It was left to Howe, following Larkin’s late lengthy miss at goal, to cut loose from 40 metres out and stun the home crowd for an 8-0 interval lead.
A needless off-the-ball altercation between O’Driscoll and Howe saw both yellow-carded by referee Alain Rolland on the restart, and Leinster were back to 8-3 down through Contepomi.
A high tackle from Neil Best on 54 minutes saw the South American make it a two-point game, but even with Leinster bolstered by their impressive replacements, eighth-placed Ulster looked like holding firm.
Contepomi had a crack from inside his own half on the hour and Paul Steinmetz drilled just wide as both sides began to tighten up.
Ulster replacement Nigel Brady then gave Contepomi another angled chance on 77 minutes, but even though the 27-year-old drilled it just wide, Frost’s late incursion saw Ulster cruelly sucker-punched at the death.