Stinson, who was recalled from Triple-A Norfolk before Wednesday's game to fill the empty starting rotation spot, was making his second career major league start, and his first since Oct. 3.
It was the first time an Orioles pitcher has allowed four homers in a game since Jason Hammel allowed four last May 30.
Stinson has mostly been used as a reliever – he recorded a 0.96 ERA in six appearances, five in relief, last season with the Brewers and was 1-0 with a 1.50 ERA in 10 spring appearances with Milwaukee – but was 1-0 with a 0.77 ERA in two starts at Triple-A Norfolk this season.
Stinson had allowed just two homers over 23 1/3 career major league innings, but the Blue Jays launched four well-hit balls into the seats at Camden Yards.
The Blue Jays then pounced on Stinson early in the count for three more solo homers.
Rajai Davis hit his first homer of the season on a 0-1 pitch in the third, while Edwin Encarnacion and Jose Bautista both homered to left on first-pitch offerings in the fourth and sixth innings, respectively.
Stinson left the game after 5 2/3 innings, allowing five runs on five hits with three strikeouts and one walk.