Leinster dominated the opening exchanges and were rewarded after eight minutes when Jimmy Gopperth kicked a straightforward penalty following an offside infringement.
Gopperth, a summer signing from Newcastle Falcons, targeted the young Ulster back three early on with a range of testing kicks but the trio of Peter Nelson, Rory Scholes and Michael Allen dealt admirably.
As Leinster continued to apply pressure, the hosts were penalised for failing to roll away and Gopperth made no mistake with his second effort to give his side a 0-6 lead after 18 minutes.
Three minutes later flanker Dominic Ryan capitalised on good work from his forwards to dive over from a rolling maul. Gopperth converted for a 0-13 lead.
Ulster responded superbly and, aided by quick breakdown service from Michael Heaney, they worked through numerous phases and earned field position inside the opposition 22.
Playing at outside centre, captain Payne cut a fantastic angle off his outhalf James McKinney and showed great strength to crash over beside the posts. McKinney, returning from a season-long spell with Rotherham, had no problem converting to leave the scoreboard reading 7-13 as the half hour mark approached.
Buoyed by that score, Ulster started to threaten more with ball in hand and earned a penalty after a half-break from Nelson. Payne made the brave call to snub the posts and kick to touch, but despite losing the initial lineout, his bravery paid off.
Just before half time, great defence by Sean Doyle forced a scrum five metres out and from there, last season's PRO12 Players' Player of the Year Williams brushed aside three defenders to score. McKinney slotted over the extras to send his side into the new changing rooms in the Aquinas stand with a 14-13 lead.
Despite holding a slight advantage, Head Coach Mark Anscombe made wholesale changes to his side at the interval. Twelve new players emerged from the tunnel for the second period, including former Coventry prop Bronson Ross and hooker Jonny Murphy, who were making their debuts for the Province. Doyle and Chris Farrell, who missed the entire 2012/13 season through injury, along with McKinney, were the only players to return to the fold.
Ulster enjoyed a healthy chunk of possession after the break but were unable to register on the scoreboard. Instead, Gopperth kicked Leinster back into the lead with two penalties, before he was withdrawn from the action on 58 minutes.
McKinney missed an opportunity to cut the arrears with a penalty but Ulster continued to push for the decisive score. In particular, the new back three of David McIlwaine, Chris Cochrane and Ricky Andrew all looking dangerous and hungry for work.
Eventually, as the full time whistle drew closer, the lively Porter, a replacement for Heaney, forced his was over the whitewash to the delight of the home fans. McKinney sealed a 21-19 triumph with his conversion to open the new Ravenhill in style.
ULSTER: (15-9): P Nelson, R Scholes, J Payne (Captain), C Farrell, M Allen, J McKinney, M Heaney; (1-8): K McCall, N Annett, D Fitzpatrick, L Stevenson, N McComb, M McComish, S Doyle, N Williams.
Replacements: Callum Black, Jonny Murphy, Bronson Ross, Alan O'Connor, Dan Tuohy, Robbie Diack, Roger Wilson, Ian Porter, Chris Cochrane, Luke Marshall, David McIlwaine, Ricky Andrew (all '40).
LEINSTER: (15-9): Darren Hudson,?Darragh Fanning, Brendan Macken, Noel Reid, Andrew Boyle, Jimmy Gopperth, John Cooney; (1-8): Jack O'Connell, Aaron Dundon, Martin Moore, Tadhg Beirne, Quinn Roux, Rhys Ruddock (Captain), Dominic Ryan, Leo Auva'a.
Replacements: James Tracy, Jack McGrath, Michael Bent, Tadhg Furlong, Ben Marshall, Jordi Murphy, Dan Leavy, Luke McGrath, Colm O'Shea, Adam Byrne, Thomas Daly, Billy Dardis, Thomas Farrell, Cathal Marsh.