WASHINGTON and SAN FRANCISCO — Think you can hack the Pentagon? Well, soon you may get the chance to crack its systems – without violating national security.

The Department of Defense announced Wednesday that it will invite vetted security researchers to test its cybersecurity systems in a pilot program set to begin next month.

It will be the first cybersecurity bug bounty program in the history of the federal government, and hackers "could be eligible for monetary rewards and other recognition," a Pentagon spokesman said, adding that the initiative is modeled after similar competitions conducted by some of Silicon Valley's biggest tech companies.

"Bringing in the best talent, technology, and processes from the private sector not only helps us deliver comprehensive, more secure solutions to the DOD, but it also helps us better protect our country," said Chris Lynch, a technology entrepreneur who directs the Defense Digital Service, which will oversee the bug bounty program.

The Pentagon made it clear, too, that approved researchers won’t be allowed to run amok in DOD computer systems, but rather will be asked to identify vulnerabilities on a "predetermined" department system.

The project is a demonstration of a drive within the Pentagon to improve cybersecurity, said Secretary of Defense Ash Carter.

"I am always challenging our people to think outside the five-sided box that is the Pentagon," he added. "Inviting responsible hackers to test our cybersecurity certainly meets that test."

News of the program came during the RSA Conference in San Francisco, the largest annual gathering of security professionals, and generated plenty of buzz among attendees.

"Even the strongest organization on the planet, essentially, even they know that to be fully safe they need to work with hackers," says Mårten Mickos, chief executive officer of HackerOne, a San Francisco firm that operates bug bounty programs.

The Pentagon's decision, said Mr. Mickos, "shows a shift from the old world of secrecy to the new world of transparency. From an old world of keeping it close to the vest, to collaborating broadly. I think it's absolutely pioneering."

The program – dubbed "Hack the Pentagon" – shows that "DOD is clearly using commercial best practices," says Ben FitzGerald, director of the Technology and National Security Program at the Center for a New American Security.

Launching an endeavor like this is not easy, Mr. FitzGerald adds. "There are significant institutional antibodies for this type of initiative."

That’s because hackers could find something embarrassing, or – despite their strict parameters – the Pentagon could be hacked in ways beyond what they might have expected. "They may like for these vetted hackers to look at, let’s say, one of the many innocuous websites they run, in the hopes that these guys will say, 'It’s really good [security]' or, 'There’s just a minor issue.' "

They may find instead, "You didn’t have any password protection, or you had 'password' as a password,' " FitzGerald says.

That’s why the new hacker initiative is an important step toward "demystifying the security needs of the [Defense] department," he adds, since "often when national security people talk about cybersecurity, it’s as if it’s something different than the network security that every business does."

While the Pentagon trades in top secret information, "a lot of what it does involves basic security as well," FitzGerald says. "We see organizations like Google and Facebook make frequent use of these programs and see great returns."

For the DOD, he added, "it's the closest you get to a Silicon Valley model of disrupting and innovating."