West Va. Internet consultant paid $512k in federal stimulus funds

Earlier this year, reports emerged that West Virginia had become the home of the $22,000 router. Using federal stimulus money, state officials decided to install more than 1,000 public institutions with enterprise-grade Cisco routers, according to reports in the Charleston Gazette. The sites included rural libraries with just a few computers, which could have been just fine spending less than $500 on a router, if not less than $100.

Now it turns out that not only are officials in the Mountain State spending big sums on high-powered routers; they're also richly rewarding the consultants in charge of planning the whole fiasco. A story published today in the Gazette reports $1.3 million has been spent on consultants hired through a contract with Verizon.

The highest-paid one, Perry Rios, was paid $196 an hour, for a total of $731,770 through the end of last month. Rios, who lives in Colorado, made $512,000 in 2011 and is on track to earn another $329,000 this year.

Rios isn't the only high-dollar consultant hired by the state. Two network engineers were paid $252,075 and $143,490 under $150-an-hour contracts. Two in-state project managers were paid $99,661 and $97,500 respectively, under $250-an-hour contracts. The five consultants were paid $1.3 million in total. All of the payroll information is based on a Freedom of Information Act request filed by the Gazette.

None of the five consultants appear to have spoken at any public meetings, including monthly meetings of the Broadband Deployment Council (BDC). Rios has met twice with the BDC's chairman, but none of the five consultants have filed any reports with those agencies. For his part, Rios also meets regularly with the "Tiger Team"—a group charged with ensuring public facilities get their upgraded routers and fiber-optic connections—and participates in conference calls with the federal agency that distributed the funds.

West Virginia officials justified the high-priced consultants by saying they were needed to oversee the massive scope of the project. The state received $126.3 million in federal funds to upgrade its broadband Internet network, getting routers and fiber-optic cable to more than 1,000 public institutions, including schools, libraries, state agencies, and health centers.

"This work goes far beyond our current staffing resources," Gale Given, chief technology officer for West Virginia, told Gazette reporter Eric Eyre. "These professionals are necessary to provide engineering, project management and other functions, and to coordinate the various parties that are involved in the grant."

Rios has a state voicemail and e-mail address, but didn't respond to calls and e-mails from the newspaper requesting comment.

The final kicker: Rios' job functions seem to be largely mirrored by another "project manager," Nick Patel, already working in a state technology office.

The Gazette's earlier reports about the pricey routers ultimately made their way to Congress, where state officials actually defended the purchases as an "economical" decision.

"All of this might be funny if West Virginia wasn't still a broadband backwater," opines Karl Bode of DSL Reports, who also blames Verizon for neglecting the state's infrastructure. "They're also the worst in the country in terms of a disconnected populace, with recent FCC data showing that 46 percent of the state's residents don't have access to broadband."

And this type of BS happens often.Bet we could shave Trillions off the Debt just over Mismanagement and Corruption.

It seems really bad because it is, but pure waste (at the federal level) only accounts for a few tens of billions of dollars a year at most, which is around NASA territory (read: barely significant). The waste is disappointing, but it's nothing next to spending on Social Security, Medicare, Medicaid, and other entitlements.

Edit: It appears that this comment is coming across as equating entitlements with waste. It is not. It's only meant to compare waste (like what's observed here), purely in magnitude, to where the government's money actually goes.

I just want to point out that you do NOT want to use a $100 Linksys in a public library setting.

At the very least, you need a router that is a little more configurable and can be centrally managed by the city or county IT staff. You also need something that can deal with content control and filtering, and you want something that's not easily hackable. Putting a cheap home router in a public facility is just asking for trouble.

This alone makes it important to go with equipment that you won't find plugged in to a home Internet connection. Whether it's Cisco or some other brand, you need something better than SOHO equipment.

Now don't get me wrong: $22,000 still sounds like overkill, but don't undermine your own credibility by pointing to something on the shelf at Best Buy and telling people that it's at all appropriate for a public facility that needs a much more powerful piece of hardware than anything the Geek Squad will install for you.

@law ss is not an entitlement unless you mean your 401k is an entitlement.

An entitlement is "a government program providing benefits to members of a specified group" (Merriam-Webster). People object to the phrase for SS because nowadays "being entitled" has negative connotations--it originally did not. And yes, if my 401k were gov't-guaranteed based on my income level (or geographic location or industry of work or military service or some other identifiable characteristic), it would be an entitlement. Something not from the government cannot, by definition, be an entitlement (in the sense that the word is used to describe SS).

And this type of BS happens often.Bet we could shave Trillions off the Debt just over Mismanagement and Corruption.

It seems really bad because it is, but pure waste (at the federal level) only accounts for a few tens of billions of dollars a year at most, which is around NASA territory (read: barely significant). The waste is disappointing, but it's nothing next to spending on Social Security, Medicare, Medicaid, and other entitlements.

It makes me cringe every time some says entitlements. Waste = Entitlements? Wow....just wow. This one makes me a little shocked at even the brazenness of it.

Medicare? Why do we have it? Cause people work their whole lives...people who are retired who now are retired. Nixon put the US on the course of insurance method of provision of health care. Insurance doesn't keep going after one quits work, so now Medicare kicks in. See how it works? It is not waste it is looking after people when they are at their weakest. There is far less waste in medicare than insured methods of provision, this has been show by study after study.

I won't respond to your Social Security being an entitlement that people have been paying into their whole lives. That one would rather have old people not have anything than the meagre money they get through SS. Have you done any research as to how much people generally make on SS. Sure their are lots of people who get it. There are many people who pay into it. But to cast it in this light is both ignorant and disgraceful. Hope you have to live in the world you envision. (that is a curse)

It makes me cringe every time some says entitlements. Waste = Entitlements? Wow....just wow. This one makes me a little shocked at even the brazenness of it....I won't respond to your Social Security being an entitlement that people have been paying into their whole lives. That one would rather have old people not have anything than the meagre money they get through SS. Have you done any research as to how much people generally make on SS. Sure their are lots of people who get it. There are many people who pay into it. But to cast it in this light is both ignorant and disgraceful. Hope you have to live in the world you envision. (that is a curse)

Whoa, slow down. I bring up the entitlement programs for the reasons given--they are the single largest chunk of government spending, and so if you want to reduce government spending by a significant amount, its entitlements you need to cut, and focussing on cutting gov't waste is pursuing a mythological promised land of all the programs we love with none of the taxes we hate that does not exist. I am not saying we should cut entitlements! I am NOT equating them with waste. I am saying that they dwarf waste in the amount of spending they represent, and so you can't cut trillions from the debt even by getting rid of all government waste.

Social Security just *is* an entitlement by the definition of the word. Calling it an entitlement does not call people who receive SS payouts "entitled parasites", in the negative connotations sense of entitled, as elements of the GOP are fond of calling most of America. It is a word that describes a program where the government gives money to people based on some classification scheme, regardless of the root source of the money. That's it.

And btw, unless the deficit hawks win and cut the whole thing or the anti-tax crazies prevent us from paying for it and the program collapses, I intend to draw on SS some day and not feel bad about it in the slightest, while still calling it an entitlement program (Medicare, too, although I hope not to draw on that one too much ). I believe in the welfare state and voted for Obama both times.

I can see a $100 Linksys router fitting the bill for things like rural libraries. However with the type of information that is transmitted/stored and the laws governing how data is protected and handled by such places, they may need something a tad more expensive to meet compliance. But that is no where near $22k, that is just crazy talk. Next they are going to find the $50+k fee just to teach or show someone how to manage/care for the router, per place, per person, 2 or more is extra.

I wonder if the contract was put out for bid like a lot of public and federal money should be.

I'm sure it was, and that is part of what increases the price. There is a lot of red tape for people in the agency to get through to buy something and a lot of red tape for consultants and vendors to get through to sell something.

A government can't just go buy from the cheapest vendor on Amazon. The laws on the books to prevent corruption can cost more than they save in some cases.

I don't know the this case well enough to say it was excessive or not, but I do know enough about government procurement to say that it isn't as obviously black and white as most people here seem to think.

I don't want to be the bearer of bad news, but those rates for IT consulting are pretty much what one would expect for IT consulting, especially if the invidual is working for a larger firm. $150 is low for anything but an IT generalist, and a subject matter expert should expect to cost anywhere from $200 to $250 per hour.

I suppose some of the shock comes from the fact that its not obvious from the reporting that this money went to the consultant's firm and not into his or her pocket. An independent consultant or small firm wouldn't have the pull or resources to secure a state contract. That means most of the $200 per hour went back in to the business.

It appears that Perry Rios and a few others are just more in the long line of professional charlatans in the IT world.

I quit my last two jobs because of them.

It's sickening when organizations pay consultants >5x the annual salary of their IT people for something their IT people could do just as well. It happens in my institution all the time, except that the consultants come up with crap that just makes things worse and walk away with gobs of money and then we have to fix this shit on our regular salaries. I should become a consultant, but I can't bullshit too well and I have morals.

JTD121 wrote:

Right, so no offense, but where do I sign up for a job that pays me $200/hour to ignore common sense and spend stupid money to connect 15 people to the internet?

It's not surprising. My mother-in-law works for a WV school system and she has to fight cronyism every day in her very low level job. The whole system is rotten from top to bottom. At least Robert Byrd would have made sure the money stayed in-state.

A good article, Mr. Mullin; however, please let me comment on a couple of items. First, we all should remember that the primary objective of ARRA was to stimulate the economy, i.e., spread the wealth around, create jobs and build essential infrastructure. Spreading it wastefully or efficiently didn’t matter, either way it infused money into the economy, albeit fiat money. The second item I would like to make a point on is that it was the grant/loan recipients’ intent, and the governments’ objective, to build necessarily, the best networks money could buy; these networks are intended to be state of the art, future proof, enterprise class networks, with an overall ongoing Lower total cost of ownership over their lifespans, and it is this TCO component that decides whether the true cost will be upfront CapEx or the much higher OpEx spread over time.

Your article, and the readers’ subsequent comments, seem to suggest that these projects should be comparatively priced to the provisioning of consumer class equipment from Wal-Mart and Best Buy by high school computer hobbyists who will also design, build, operate and maintain the systems. Building an enterprise class network for essential and necessary public services is not the same as building a home network; as the old saying goes, “you will get what you pay for”. It further reminds me of a quote someone once said, “if you want to bark with the big dogs, you have to come down off the porch”; you simply cannot build fault tolerant redundant first responders’ systems of the caliber that ARRA was intended to build, with toy walkie talkies.

The consultant fees and equipment prices cited in your article are on par with the recipients’ and governments’ aforementioned objectives. If it were the consultants’ intent to gouge the taxpayer, we certainly could have done so by simply increasing the total cost of ownership (over the life of the network) by using $ 100 consumer class routers rather than enterprise class equipment et. seq. The networks could have been designed to ensure endless recurring revenue to the consultants’ in the form of tech support fees, maintenance and repair fees, annual upgrades et. al. The cheaper folks will always cost you more in the long run.

Many of these ARRA funded networks are to be used for 24/7 mission critical communications and applications. If you’re having a heart attack and your 911 VoIP call fails because of a cheap router that can’t be monitored, diagnosed, and rebooted from a NOC, then enterprise class begins to sound more reasonable. As for the library example, I use the streaming media our local library serves up, its a new educational tool, and libraries are good places to implement telepresence for everyone, new business, education; I get it, it’s about the future and the bandwidth demands that are rapidly increasing every day.

I am probably the last person on earth to condone the government funding of anything, but when people start condemning a wasteful government program, as being well, wasteful, that tells me that the people complaining have perhaps lost sight of the original objectives of the program, but also more obviously, they are not comparing apples to oranges. The complaints about the hourly fees are without merit, you cannot compare the hourly rates of these self employed, independent consultants, to those hourly costs of in-house employees with similar job titles; there are too many variances relating to insurance, taxes and similar costs to compare them without also addressing these additional costs that employers ordinarily bear. My colleagues and I are not wasting taxpayer money, we were engaged to develop the best networks that your money can buy and it would be suicide to our respective individual reputations to build faulty networks that, upon failure, will be immediately in the public eye, particularly when many peoples’ lives will be relying on these systems.

Sorry. Some of those guys may be overpaid. But if they have the credentials to back them, the guys in the 90's and 150's are getting paid market rate for something of that scope. In the article it mentions hooking up library's, schools, and other government buildings to fiber lines.

Networking for one building is it's own set of problems. A program like this practically requires you bring in consultants of that caliber. This isn't something you DIY to save a couple bucks. I can it imagine it turning out drastically worse if it were left to public servants.

...we all should remember that the primary objective of ARRA was to stimulate the economy, i.e., spread the wealth around, create jobs and build essential infrastructure. Spreading it wastefully or efficiently didn’t matter, either way it infused money into the economy...

A foolish comment. If it didn't matter how the money was spent, a lottery would have been just as effective. Clearly the money was meant to be spent efficiently and wisely.

Quote:

Building an enterprise class network for essential and necessary public services is not the same as building a home network; as the old saying goes, “you will get what you pay for”. It further reminds me of a quote someone once said, “if you want to bark with the big dogs, you have to come down off the porch”; you simply cannot build fault tolerant redundant first responders’ systems of the caliber that ARRA was intended to build, with toy walkie talkies.

Another bit of foolishness, with a straw man to boot. No one suggests they need "toy walkie talkies" -- just that they WAY overpaid for what they purchased. The Cisco routers for my university didn't cost anywhere near $22k. 4 years later, all the cost involved wouldn't put them at half that amount.

Quote:

The consultant fees and equipment prices cited in your article are on par with the recipients’ and governments’ aforementioned objectives. If it were the consultants’ intent to gouge the taxpayer, we certainly could have done so by simply increasing the total cost of ownership (over the life of the network) by using $ 100 consumer class routers rather than enterprise class equipment et. seq.

I've been a consultant. If a job is going to take more than a few hundred hours (or thousands, as this one apparently did), I tell the client that their money is better spent hiring a salaried employee on a year (or multi-year) contract. To do otherwise would be dishonest.

Quote:

Many of these ARRA funded networks are to be used for 24/7 mission critical communications and applications. If you’re having a heart attack and your 911 VoIP call fails because of a cheap router that can’t be monitored, diagnosed, and rebooted from a NOC, then enterprise class begins to sound more reasonable.

Another straw man. Mission critical systems aren't what we're talking about here. One would assume that these are already *somewhat* in place, unless the good people of West VA are living a 3rd World existence (and no, slow connectivity doesn't mean 3rd World -- I've been to West VA).

Quote:

As for the library example, I use the streaming media our local library serves up, its a new educational tool, and libraries are good places to implement telepresence for everyone, new business, education; I get it, it’s about the future and the bandwidth demands that are rapidly increasing every day.

Still not an excuse for massively overpaying for routers. West VA is primarily rural. No way in hell do they need enterprise class routers, even for streaming media (which is easily handled by a $5000 router even with dozens of users at once, let alone dozens per day or per week).

I wonder if the contract was put out for bid like a lot of public and federal money should be.

I'm sure it was, and that is part of what increases the price. There is a lot of red tape for people in the agency to get through to buy something and a lot of red tape for consultants and vendors to get through to sell something.

A government can't just go buy from the cheapest vendor on Amazon. The laws on the books to prevent corruption can cost more than they save in some cases.

I don't know the this case well enough to say it was excessive or not, but I do know enough about government procurement to say that it isn't as obviously black and white as most people here seem to think.

So this is okay with you? You make it sound like this is completely reasonable. Since it's a government program it's okay to have additional layers of red tape to justify costing more money than we reasonably think it should?

...we all should remember that the primary objective of ARRA was to stimulate the economy, i.e., spread the wealth around, create jobs and build essential infrastructure. Spreading it wastefully or efficiently didn’t matter, either way it infused money into the economy...

A foolish comment. If it didn't matter how the money was spent, a lottery would have been just as effective. Clearly the money was meant to be spent efficiently and wisely.

Quote:

You totally missed the point.

Quote:

Building an enterprise class network for essential and necessary public services is not the same as building a home network; as the old saying goes, “you will get what you pay for”. It further reminds me of a quote someone once said, “if you want to bark with the big dogs, you have to come down off the porch”; you simply cannot build fault tolerant redundant first responders’ systems of the caliber that ARRA was intended to build, with toy walkie talkies.

Another bit of foolishness, with a straw man to boot. No one suggests they need "toy walkie talkies" -- just that they WAY overpaid for what they purchased. The Cisco routers for my university didn't cost anywhere near $22k. 4 years later, all the cost involved wouldn't put them at half that amount.

Quote:

The Cisco routers did not cost $ 22k, do some research.

Quote:

The consultant fees and equipment prices cited in your article are on par with the recipients’ and governments’ aforementioned objectives. If it were the consultants’ intent to gouge the taxpayer, we certainly could have done so by simply increasing the total cost of ownership (over the life of the network) by using $ 100 consumer class routers rather than enterprise class equipment et. seq.

I've been a consultant. If a job is going to take more than a few hundred hours (or thousands, as this one apparently did), I tell the client that their money is better spent hiring a salaried employee on a year (or multi-year) contract. To do otherwise would be dishonest.

Quote:

You 'were' a consultant? But you aren't a consultant any longer? This puts your entire post in perspective now.

Quote:

Many of these ARRA funded networks are to be used for 24/7 mission critical communications and applications. If you’re having a heart attack and your 911 VoIP call fails because of a cheap router that can’t be monitored, diagnosed, and rebooted from a NOC, then enterprise class begins to sound more reasonable.

Another straw man. Mission critical systems aren't what we're talking about here. One would assume that these are already *somewhat* in place, unless the good people of West VA are living a 3rd World existence (and no, slow connectivity doesn't mean 3rd World -- I've been to West VA).

Quote:

I'm building one of these mission critical systems. You should look into 911/first responders networks before you stick your foot in your mouth. Has the mission critical, FirstNet, installed its first antenna yet?

Quote:

As for the library example, I use the streaming media our local library serves up, its a new educational tool, and libraries are good places to implement telepresence for everyone, new business, education; I get it, it’s about the future and the bandwidth demands that are rapidly increasing every day.

Still not an excuse for massively overpaying for routers. West VA is primarily rural. No way in hell do they need enterprise class routers, even for streaming media (which is easily handled by a $5000 router even with dozens of users at once, let alone dozens per day or per week).

Shill elsewhere.

Quote:

How do you know what kind of router is needed anyway, did you do their needs analysis?

Quote:

Do you really feel this insecure, that you have to call me foolish, and try to discredit every point with condescension? Can't you engage in honest, productive dialogue and debate, without sinking to the Jerry Springer Show level?

I just want to point out that you do NOT want to use a $100 Linksys in a public library setting.

With rural libraries with just a few computers? I think that is exactly what you DO want. $22k could be the budget for library's staff for a year.

A rural library with just a few computers is exactly where you want something enterprisy. The alternative is to either hire someone local to do the admin stuff or send someone out every so often to do it. Both of those options are way more expensive than these overpriced routers.

I wonder if the contract was put out for bid like a lot of public and federal money should be.

I'm sure it was, and that is part of what increases the price. There is a lot of red tape for people in the agency to get through to buy something and a lot of red tape for consultants and vendors to get through to sell something.

A government can't just go buy from the cheapest vendor on Amazon. The laws on the books to prevent corruption can cost more than they save in some cases.

I don't know the this case well enough to say it was excessive or not, but I do know enough about government procurement to say that it isn't as obviously black and white as most people here seem to think.

So this is okay with you? You make it sound like this is completely reasonable. Since it's a government program it's okay to have additional layers of red tape to justify costing more money than we reasonably think it should?

Not at all. I'm saying it is possible that the people in this case acted reasonably with the set of rules they were required to follow by law.

Lets just pretend for a minute that they had found and used a vendor on Amazon that could supply the routers more cheaply, all things being equal. They could very well have opened themselves up to a lawsuit for not using a vendor on an approved contract, or because it wasn't "competitively bid". Even if your worst case is just re-bidding the contract, you still wasted a lot of time/resources internally and externally and you've royally pissed of your purchasing department because you didn't listen to them when they told you not to do that. Good luck with your future purchases.

Furthermore, some money can ONLY be used to buy equipment and services. It isn't surprising then that consultants are hired to do the design. Governments tend to only create positions for long-term operations staff, leaving design and architecture to consultants because there isn't an ongoing need for those capabilities in house.

I promise you that most government workers involved in procurement, from a teacher buying classroom supplies to an IT director, find the rules of the game wasteful. It runs counter to how they make their own purchasing decisions at home.

I often wonder if the rules cost more, in the aggregate, than they save.

Um, I think somebody's having trouble with the 'quote' functionality. Made this a wee bit hard to read.

Also, sounds like a bit of 'doth protest too much.' Granted, I've never been out on this sort of job (for that matter, I hate networking with a passion), but I'd be interested to see an OGA-style analysis on this project. Any project with this kind of money has some waste and corruption. The question is, how much?

I don't want to be the bearer of bad news, but those rates for IT consulting are pretty much what one would expect for IT consulting, especially if the invidual is working for a larger firm. $150 is low for anything but an IT generalist, and a subject matter expert should expect to cost anywhere from $200 to $250 per hour.I suppose some of the shock comes from the fact that its not obvious from the reporting that this money went to the consultant's firm and not into his or her pocket. An independent consultant or small firm wouldn't have the pull or resources to secure a state contract. That means most of the $200 per hour went back in to the business.

So I do like most "consultants", and set up a company: "Fred Smith GROUP Incorporated (Delaware)". Big myself up (even though I'm a one-man business). And then start charging $200 per hour for work that should come in at ≤$500 per full day in the real-world...What do you think? That it costs 3× as much to run a "business" as it does to employ a person? Businesses are people, my friend—cuts both ways!

I don't want to be the bearer of bad news, but those rates for IT consulting are pretty much what one would expect for IT consulting, especially if the invidual is working for a larger firm. $150 is low for anything but an IT generalist, and a subject matter expert should expect to cost anywhere from $200 to $250 per hour.I suppose some of the shock comes from the fact that its not obvious from the reporting that this money went to the consultant's firm and not into his or her pocket. An independent consultant or small firm wouldn't have the pull or resources to secure a state contract. That means most of the $200 per hour went back in to the business.

So I do like most "consultants", and set up a company: "Fred Smith GROUP Incorporated (Delaware)". Big myself up (even though I'm a one-man business). And then start charging $200 per hour for work that should come in at ≤$500 per full day in the real-world...What do you think? That it costs 3× as much to run a "business" as it does to employ a person? Businesses are people, my friend—cuts both ways!

You can setup a one man band business and perhaps your reputation/skill/presence in the market will let you charge those rates. That's certainly possible and well established consultants do that. However, many contracts demand the backing of a substantial and credible company. When you start down the road of growing your one-man-band company into a substantial and credible company, you discover that it can easily cost 3x to run a business compared to employing oneself