Britain's digital spies have turned to Google for help making sense of the floods of data now inundating their powerful computing resources.
GCHQ, the Cheltenham-based signals intelligence agency, is recruiting an expert on MapReduce, the patented number-crunching technique previously behind the dominant web search engine.
The …

"don't be evil"

31K for a lead developer

From the ad it sounds like they are prepared to accept recruits straight out of uni - for a "lead/expert" role - for ~30K and a basic pension when google finance and bio offer betweern 45 and 60K (or more if you are an "expert"). And anyone who thinks joining as a specialist will be a career path needs to talk to a good shrink or recruiter.

Given the salary is the idea to dissuade qualified candidates so they can apply to bring some already hired indian PhD grads in for 30K or less as there are no UK IT specialists available.

LOL...

BigTable?

I think not - they always used BigTable in conjunction with Map-Reduce. What they have developed is a less batch driven version of their indexing MR jobs and adopted some new fancy tech that we do not know a lot about yet apart from it indexes faster...

Does that mean MR is inherently poor for data analysis? No - it means its not up to the job of creating the large index Google needs very quickly to keep their results fresh. Batch analysing large data sets is still a good use-case for MR and other large corporate companies such as IBM, HP etc are investing heavily in it.

GCHQ? - isn't that....

Aw, ain't that cute.

They're trying to build an NSA type snooping thing on the cheap.

But how much of all the possibly thread potential indicative data the NSA and its TLAgency brethren does it actually make into cogent reports? Well, you could argue the inherent goodness in that, but then there's the next bit: How many of those reports with Vital Information do make it to people who Need To Know? Ah.

And on how much of THAT is actually acted upon? In a useful manner?

Four aeroplanes and a pants bomber tell me: Not enough.

So, is that going to work out better by going the same route with less preparation and on a smaller budget? Will this enable them now to tell which toner cartridge has a detonator mobile phone hid inside it before it reaches the airport?

Is really that painful for our spies that it's too much to ask to go back to HUMINT again?

Cheltenham ..... we have a problem.

"This is the problem with politicians. .... The filthy thieving scum walk of Parliament with box cars full of swindled money, yet expect 190 IQers to work for less money than it costs to hire a full time Nanny after tax.

Surely what is even more ludicrous and quite ridiculous, is that the myriad supposed Intelligence Services and JIC type quangos and GCHQ allow all the crooked shenanigans and City frauds and banking scams on their watch. It proves that they have vast swathes of Missing Intelligence and are themselves as useful as the hopeless MOD.

Where is the System which Leads for Others to Follow and React to ....... rather than that as is now, with slippery tongued politicians in Wannabe-in-my-Gang Banging Government begging for scraps abroad in Smarter Valuable Foreign Fields.

Is it to be Right Royally Provided from the Private Sector in a Rogue IT Initiative with Immaculately Resourced Assets plucked from the Ether and Tendered in the Ilk of an HyperRadioProActive Office of Cyber Security? Or does the bankrupt Public Sector imagine itself to have everything adequately covered and everything under its IT Control, although methinks that is more pantomime than circus, don't you, and a delusional madness which is rendering third world pariah status to a once rightly proud and benevolently imperial nation?

But don't rely on incomeptance to not get the job done.

They might still hire someone good enough *despite* the relatively poor pay.

This is still expected to be the UK *largest* government IT project *ever* (NHS IT started at £5Bn. It grew) The only *full* figure ever released has been £12Bn (and that was probably before Dettica gave them the full price list on their snoop boxes with *all* the trimmings).

Damned IMPertinent Virtual Cheek ....... Now there is no excuse for pretending that nobody knows

JS 19, What they may be lucky/astute/smart enough to do is hire somebody who can generate billions with AIMODified Systems which make much cleverer use of compromised assets, which would be both theirs at home and others abroad, for that would be a far better plan than to be facing a competitor better equipped and who speaks their language quite well enough to make Future Building interesting, for the following Work in Progress is currently Live Beta Testing the Operational Virtual Environment for ....... well, Takers rather than Losers, Movers and Shakers in the Field which is chock full of Wimps and Wallies in Office.

[quote]Competition in the Defence Weapons Systems Arena, necessarily will/may require that an overwhelming attack against established programs and protocols be exercised, as a live demonstration of new proposal capability and easy facility, should a hitherto quite unknown and stealthily, underground tested system, suddenly appear and be recognised as being a viable and vitally important component for future national security and international and InterNetional Defence, both strategic and tactical. And that creates an abiding and challenging active enigmatic dilemma for both the novel weapons systems suppliers/vendors and the customer, for once such a vulnerability is shown to exist and to be easily exploited/exploitable to render all manner of attack successful, [and in the best of any novel weapons systems, any current defence capability totally ineffective], is both the military and intelligence integrity of the customer nation, and its future defence and attack capability in what may be/will be/is a New Field and NeuReal SMARTer Intelligence Front, demonstrated to be catastrophically compromised and any earlier perceived notion of superiority and shock and awe pimped and pumped authority in what is a Global Command and Control Space and Live Operational Theatre/HyperRadioProActive Medium, immediately lost, and the value and worth of the new proposal and System, priceless, and obviously an attractive proposition and system to other nations/nationals who may have an interest in, or be competing for overwhelmingly advantageous leverage in Advanced IntelAIgent Virtual Defence Force Fields and CyberIntelAIgent Security Operations Systems.

And the RAND corporation would appear to agree on the enigmatic dilemma .... "In some cases—especially in the procurement of major systems whose nonrecurring costs are large—it may actually be less costly for the government to forgo competition." ..... http://www.rand.org/pubs/monographs/2010/RAND_MG1020.pdf [Page 16/159] ...... although, the cost of a new procurement of a major system which can render an overwhelming advantage in say, the Command and Control of Virtual Assets and IntelAIgent Machines in CyberSpace Space, and which may be a priceless asset to have in one's portfolio/arsenal is always, surely, subjectively decided on by the customer and will be relative to their need for its feed, and not by the vendor at all, who would know exactly its worth and value to any and all customers, whether domestic or foreign, military or civil [/quote] ....... which is quite enough of that for now, methinks.