Lester was born in a log cabin in Minnesota. His parents were John Porter Armstrong and Louise McDowell Armstrong. He had a twin brother, Leslie, and also had twin siblings, Edna Belle and Harry Joe, and two sisters, Laura A and Frances (Franc) May.