Milwaukee appeared to have won the game on Greg Monroe's last-second hook shot, giving the Bucks a 111-109 lead. Olynyk then drew the foul with no time expiring off the clock.
Monroe, who was removed from the starting lineup, had 29 points and 12 rebounds as the Bucks ended a five-game losing streak. Middleton had 20 points and Michael Carter-Williams, who also was benched, added 16.
Jae Crowder and Bradley had 18 points each for the Celtics.
After trailing by a point at halftime, Milwaukee built a 19-point lead after three, sparked by late 3-pointers by Middleton and Jerryd Bayless and a buzzer-beating, left-handed hook shot by Monroe.
Boston, which had won nine of 10, chipped away at the lead throughout the fourth before Crowder hit a 3-pointer with 22 seconds left to tie it.
Monroe entered the game early in the first quarter, and Carter-Williams followed a few minutes later. Both had an almost immediate impact.
The Bucks grabbed their first lead at 49-46 on Rashad Vaughn's 3-pointer with just less than five minutes remaining in the first half. The Celtics led 54-53 at the half.
The Celtics built a 10-point lead midway through the first quarter and led 30-26 after a period. Boston shot 57 percent in the quarter.
Celtics: Crowder, who has been hobbled by an ankle injury, was in the starting lineup. … Boston shot 50 percent overall in the first half but just 5-of-15 from 3-point range.
Bucks: Coach Jason Kidd on the lineup change: "It's nothing that Michael or Moose (Monroe) have done, but we're just going to try something different. Here at the 53-game mark we'll look at a different lineup. We need to find out about some different guys going forward. Our record isn't where we want it to be." … Mayo returned after missing 11 games with a left hamstring injury. He missed 11 games earlier in the season with a right hamstring injury.