Whisk milk and potato flakes together in large bowl. Add beef, 1/2 tsp. salt, and 1/2 tsp. pepper and knead until combined. Form meat mixture into four 1/2-inch thick oval patties and transfer to parchment-lined plate. Refrigerate patties for 30 minutes or up to 4 hours.

Melt 1 tablespoon butter in large nonstick skillet over medium-high heat. Cook patties until well browned, about 5 minutes per side. Transfer patties to plate and tent with foil.

Add onion and remaining butter to now empty skillet and cook until onion is softened, about 5 minutes. Add mushrooms and 1/2 tsp. salt and cook until liquid has evaporated, 5 to 7 minutes. Stir in tomato paste and flour and cook until browned, about 2 to 3 minutes. Slowly stir in broth and port and bring to simmer. Return patties to skillet, cover, and simmer over medium-low heat until cooked through, 12 to 15 minutes. Season gravy with salt and pepper to taste.