After Latics defended large parts of the game well, the deadlock was broken on 72 minutes as Sharp converted on the volley from inside the penalty area.
In the final minute of the game, Sharp converted his second of the afternoon, despite what looked a foul on Connor Ripley as Latics came away from Bramall Lane pointless.
Manager Stephen Robinson made three changes to the side which played Southend United last week.
One of those changes, Darius Osei, started the game brightly and had Athletic’s first chance of the game.
The striker created space for himself on the edge of the penalty area but his shot was comfortably saved by Blades stopper Simon Moore.
The goalkeeper had to be alert two minutes later as Latics almost took the lead through former Sheffield United midfielder Ryan Flynn.
It was from a knockdown in the penalty area which created the chance, Flynn struck sweetly on the volley but Moore managed to tip his effort over the crossbar.
It was a positive start from Robinson’s men who were taking the game to Chris Wilder’s Sheffield United side.
However, the home side did start to come back into the game and looked threatening in attack.
On 17 minutes, after a goal mouth scramble, Charles Dunne was forced to clear twice in front of goal after Matty Done struck goalwards.
But Latics always looked promising on the counter attack, despite Flynn being booked midway through the first half for a foul on Kieron Freeman.
The hosts really should have taken the lead on the half hour mark as Billy Sharp peeled off Dunne at the back post but his header hit the defender and fell wide of the post.
Peter Clarke brought down Done 25 yards from goal but Mark Duffy’s effort was well saved by goalkeeper Connor Ripley.
The Latics number one was forced into another stop minutes later, as Jack O’Connell’s far post header was saved well.
Billy Mckay almost capitalised on a mix up in the Blades defence five minutes before the break but O’Connell managed to get back and deny the striker.
Mckay had one final opportunity before the break, volleying goalwards inside the area but his shot went over the crossbar.
Sheffield United had the first chance of the second half but Dunne did well to block Duffy’s effort from 18 yards.
Latics were defending resolutely and stopping Sheffield United creating clear cut chances.
The Blades brought on Leon Clarke and on the hour mark, he challenged Dunne in the penalty area but it was the Latics man who managed to clear the danger.
Whilst Athletic were forced into defending, they did look promising on the counter attack. Flynn nicked the ball from the Blades defenders, fed Mckay but his shot was blocked.
Sharp went close firing over from the edge of the box minutes later but on 70 minutes, Latics had a good opportunity from 25 yards.
Paul Green was fouled and Josh Law stepped up but his free kick which was towards the top corner was just too high and went over the crossbar.
Two minutes later, however, Sheffield United did manage to take the lead through Sharp.
The striker pounced on a through ball into the penalty area and volleyed into the far bottom corner past Ripley to give the Blades the lead.
Robinson made a change and brought on Freddie Ladapo and five minutes after his introduction he almost found the leveller.
The ball sat up perfectly for him 25 yards from goal and he aimed towards the top corner but Moore scrambled across to save.
Duffy forced Ripley into a smart save on 83 minutes but the Blades added a second before the end of the game as Sharp grabbed his second.
There was a huge hint of a foul on goalkeeper Ripley as he and Clarke came together, however, the referee saw no foul and Sharp converted the loose ball from 12 yards.
Clarke could have converted a third for the Blades in additional time but he struck over from 12 yards as Latics came away from Bramall Lane without any points.
Sheffield United: Moore, Fleck, O’Connell, Basham, Sharp, Done (Clarke 53’), Coutts (Wright 90’), Freeman, Ebanks-Landell, Duffy (Scougall 90+4’), Lafferty.
Subs not used: Lavery, Ramsdale, Brown, Brooks.
Oldham Athletic: Ripley, Law, Clarke, Wilson, Dunne (Croft 80’), Dummigan, Green (Winchester 88’), McLaughlin, Flynn, Mckay, Osei (Ladapo 73’).
Subs not used: Kettings, Burgess, Edmundson, Banks.
Goals: Sheffield United (Sharp 72’, 89’) 2-0 Oldham Athletic
Attendance: 25,821 (538 visiting supporters)