Discard fat from skillet and reduce heat to medium-low. Add almonds and remaining 2 tablespoons butter and cook, stirring, until almonds and butter are golden brown, 1 to 2 minutes. Remove from heat and stir in lemon juice. Season with salt and pepper. Pour sauce over fish and Brussels sprouts.