Yankees starting pitcher David Phelps reacts to a balk called on him in the third inning of a game against the Milwaukee Brewers on Sunday, May 11, 2014, in Milwaukee. Photo Credit: AP / Jeffrey Phelps

Latest Yankees stories

Nor, for that matter, did the bullpen, called on too frequently in the middle innings of late.

Phelps could not hold an early three-run lead and Adam Warren could not keep it tied late as the Yankees fell to the Brewers, 6-5, in front of 43,544 at Miller Park.

With the Yankees a strike away from a 5-4 loss in the top of the ninth inning, Mark Teixeira's two-out homer off Francisco Rodriguez tied the score. But Mark Reynolds, a Yankee the second half of last season, drove in Rickie Weeks with a two-out single in the bottom of the ninth to win it, giving the Yankees (19-17) a 3-3 record on a trip that began in Anaheim.

"They're all tough to take,'' said Joe Girardi, whose club lost two one-run games here after holding leads and, in Monday night's 4-1 trip-opening loss to the Angels, walked six batters in the deciding eighth inning.

"We lose two tough games here and you look at the trip, we lost three tough games. We end up 3-3 and easily could have been 4-2, 5-1, maybe even 6-0. And that's the frustrating part. But the guys are playing hard.''

The walk-off loss was especially stinging because of what preceded it. Against Brewers closer Rodriguez, 15-for-15 in save chances coming in, Teixeira belted a full-count changeup into the second deck in rightfield for his seventh homer, tying it at 5-5.

"He threw me a lot of good pitches that at-bat and I kept battling,'' Teixeira said. "A couple curveballs, a couple fastballs, a couple changeups, and all three are in play. I knew 3-2, he could throw any of those pitches, and I just stayed on a changeup and hit it good . . . We were feeling really good about ourselves for just a few minutes there, but that's baseball. It happens.''

Weeks, a pain all series for the Yankees (5-for-9, three RBIs), started the winning rally with one out with a broken-bat opposite-field double down the first-base line, a ball the Yankees initially thought was foul.

With 2013 Yankee Lyle Overbay at the plate, a wild pitch moved Weeks to third, but Warren responded with a strikeout. Warren got ahead of Reynolds 0-and-2, but instead of throwing a curveball in the dirt, which was his intent, he hung one, and the result was Reynolds' winning single to left.

"I get 0-2 and couldn't bury that last pitch like I should have,'' said Warren, who has allowed four earned runs in 222/3 innings for a 1.59 ERA this season. "It's tough . You want to go out and put up that zero.''

Phelps was kicking himself after being unable to hold on to the 3-0 lead provided in the first. Kelly Johnson's two-run double highlighted the inning and Yangervis Solarte hit a sacrifice fly for the first of his two RBIs. He has a team-best 20.

But the Brewers scored two in the third to make it 3-2, and after Phelps gave up two singles to begin the sixth, Matt Thornton allowed those two runners and one of his own to score for a 5-3 Milwaukee lead.

"I have to do a better job at making adjustments in the game,'' Phelps said.