The only goal of the game came on 47 minutes as Matty Done received space in the centre to finish into the bottom corner.
The Blades were reduced to ten-men on 65 minutes as Che Adams received his marching orders for pushing Liam Kelly in the face.
However, as much as Latics huffed and puffed, they could not get the winner and were knocked out of the competition.
Manager David Dunn said last week he would ring the changes ahead of this game and he did, making four changes, with Carl Winchester, Timothee Dieng, Mike Jones and Jake Cassidy all coming into the fray.
Latics had to be wary in the first minute of the game as Matty Done struck inside the box but goalkeeper David Cornell strongly palmed the ball wide of his goal and Latics cleared the danger.
Athletic started to settle into the encounter and they were the next to go close in the game.
Mark Yeates’ free kick found Danny Philliskirk free in the centre but the forward hit the post from close range.
Latics were forced into some defending afterwards but Cornell was doing his best to stem the flow of attacks.
Yeates and Winchester went close before the half hour mark with shots from the edge of the box.
Cassidy, who was making his first start since August, was putting himself about in attack, winning headers and holding up the ball intelligently.
With 10 minutes of the first half remaining, Liam Kelly opened fire from 35 yards but his shot failed to trouble Sheffield United goalkeeper, George Long.
Minutes before the break, Mike Jones’ free kick caused a problem inside the Sheffield United box, but the referee pulled the ball back after Bob Harris clutched his head in the area.
After a lengthy stoppage, Harris was taken from the pitch on a stretcher and was replaced by Kieran Wallace as the referee blew for half time.
Dunn’s men needed a lift in the second half but were hit by a sucker punch at the start of the half as they fell behind.
Neil Collins’ ball found Done and despite the best efforts from the defence, he wriggled away and slid the ball into the bottom corner to put United ahead.
After a stoppage in play as Kelly received treatment for a head injury, Latics regrouped and on the hour mark went close.
Brian Wilson’s cross into the box was punched clear by Long as far as Jones. He struck but Long, who was way out of his goal, managed to save.
On 65 minutes came a talking point as Sheffield United winger Adams received his marching orders for clashing with Kelly.
That gave Latics an impetus to go forward and Dunn brought on Dominic Poleon to add something else in attack.
The striker made an instant impact, shooting from 25 yards but a deflection saw the ball roll through to Long.
Athletic’s best chance of the game came on 75 minutes through Cassidy.
Mills’ cross into the box fell straight onto the head of the striker, who did everything right, but his header bounced onto the turf and went inches over the crossbar.
That chance inspired Latics and minutes later Timothee Dieng struck from 20 yards which the goalkeeper fumbled wide.
With the momentum and pressure with them, Sheffield United did their best to slow the game down best he could.
Latics felt aggrieved with less than 10 minutes on the clock as Poleon looked to be hauled over in the penalty box by the goalkeeper but the referee was unmoved.
United could and maybe should have been two goals ahead on 86 minutes but John Brayford’s shot from the edge of the box bounced off the bar and down, with the linesman saying the ball did not cross the line.
Heading into additional time, James Wilson tried his luck from 30 yards and despite constant pressure, Latics were defeated and knocked out of the FA Cup.
Sheffield United: Long, Harris (Wallace 45+2’), Brayford, Basham, Coutts, Sharp (Scougall 46’), Done (Sammon 84’), Collins, McEverley, Adams, Hammond.
Subs not used: Howard, Flynn, Reed, McGahey.
Oldham Athletic: Cornell, Dieng, B.Wilson, J.Wilson, Mills, Kelly, Jones, Winchester, Yeates (Rasulo 77’), Philliskirk (Poleon 71’), Cassidy.
Subs not used: Coleman, Eckersley, Vassell, Edmundson, Tuohy.
Goals: Sheffield United (Done 47’) 1-0 Oldham Athletic
Attendance: 6,938 (682 visiting supporters)