VIRGINIA BEACH — After 15 years of operation, Hampton Roads-based TowneBank is eyeing expansion to Richmond and Raleigh, N.C., to ensure future growth.

"Those markets offer great potential," Chairman and CEO G. Robert "Bob" Aston said during Wednesday's TowneBank shareholders meeting at the Virginia Beach Convention Center. "The question is, how do we do that?"

TowneBank is already becoming known in those areas because of its mortgage and insurance offices but has yet to bring banking. Because of the recent Union First Market and StellarOne merger, TowneBank was able to scoop up former StellarOne and longtime SunTrust executive Pat Collins to lead TowneBank Richmond, Aston announced.

The region's largest community bank wouldn't likely pursue expansions in both Raleigh and Richmond at the same time, Aston said in an interview. He told hundreds of shareholders in attendance that the expansion will take money, but the company hasn't identified an exact way to enter the new markets yet, whether that includes acquisitions or starting from scratch. Aston reminded shareholders to think long-term and showed them a chart in his presentation that revealed $24 million in capital available — after dividend payments, community contributions and providing employee profit-sharing from $41.4 million in profit in 2013 — to fund $300 million of asset growth.

Since the bank serves more of a niche with fewer but larger branches representing about $100 million in assets, expanding into other metro areas makes sense, Aston said. But just like how TowneBank started, he said an expansion is going to take quality people, including the next generation of shareholders, which could include some institutional investors.

Aston rallied 13 other founding directors who worked for a year in his Portsmouth garage, replete with folding tables, to get TowneBank started, he recalled. The company attracted 4,000 initial shareholders who contributed $50 million. Fifteen years later, TowneBank counted assets of $4.78 billion as of March 31 and had nearly 16 percent of the Hampton Roads' market share with $3.4 billion in deposits. Its competitors are Wells Fargo , SunTrust , BB&T and Bank of America .

At its start in 1999, TowneBank had 15 directors and 90 employees. Today, TowneBank has 9,129 shareholders, 213 directors, 1,787 employees and more than 58,800 customers, Aston said, explaining the bank grew by bringing in one person as a time. Over the past 15 years, the company was able to pay shareholders total cash dividends of $79.3 million, he said.

"It's not an insignificant sum of money," Aston said.

TowneBank operates 27 banking sites. A smaller branch is slated to open in Wards Corner in Norfolk on June 12 with a Ghent branch opening in 2015. The company plans to relocate the York office along the northbound side of Route 17, TowneBank Peninsula/Williamsburg president Brian Skinner said. The company's goal is to get to $50 million in annual earnings and Aston anticipates coming closer to that goal with the acquisition of Southern Insurance Agency in North Carolina announced in May.