[cryptome] Re: NYC Taxicab Log Dump

From: Aftermath <aftermath.thegreat@xxxxxxxxx>

To: "cryptome@xxxxxxxxxxxxx" <cryptome@xxxxxxxxxxxxx>

Date: Thu, 26 Jun 2014 13:06:19 -0700

OK :) and no, Ill have to look into Le Carre's novels.
And I prefer William ;)
On Thu, Jun 26, 2014 at 12:31 PM, doug <douglasrankine2001@xxxxxxxxxxx>
wrote:
> Dear Aftermath...or is it Bill,
>
> One of the things that I learned early in life, particularly when I moved
> to London and then to the Isle of Wight, was that taxi drivers are a mine
> of information on the local environment. When one is contemplating moving
> into a new area, then what can be better than entering into conversation
> with one's taxi driver to find out who is who and who does what in the
> local environment. Of course, one doesn't have to enter into conversation
> with one's cabbie, after all, one is a captive audience, one just has to
> guide the conversation with a few "open" questions which helps to ascertain
> the kind of environment and type of people one is going or hoping to meet
> up with if one settles there. The nice thing about such open source
> messages and conversations is that they, up until now, is that they were
> private and not recorded; and can establish so much of what one wants to
> know, without any additional expense rather than the hire. Of course,
> nowadays, the metadata is just as important...(if not more so)...than the
> content of the conversation, but that is the internet and all this new
> information technology for you.
>
> When I visited New York for a holiday...before 9/11, I met up with a
> cabbie whose English was very poor. He was a Russian immigrant, yet knew
> his way around New York so well, and was able to tell me and me missus the
> best places to go and which places to avoid and at what time. All it cost
> was a nice tip at the end of the journey, with which he was very pleased,
> and he could see that us Scots were not at all prejudiced about his origins
> or his politics. What more can one ask for in life...sarcasm is such an
> ugly word...to be accused of using it is so hurtful... ;-) .
> ATB
> Dougie.
> P.S. Have you ever read any of Le Carre's novels? He has written 23 and
> like all good security and intelligence buffs, I have read, nay, studied
> all of them. You can find them in all good bookshops and they are
> available on line very cheap. Like le Carre, I find fiction far more
> educational than non-fiction. And fiction has the added advantage that it
> is not classified or subject to secrecy laws, yet contains all the craft of
> the trade of information. Beware of course, that le Carre's main theme, as
> is that of William Boyd, is betrayal...
>
>
> On 26/06/14 19:54, Aftermath wrote:
>
> ... cant tell if sarcasm or not..?
>
>
> but yeah, I just found the link and I thought this list would appreciate
> the share :)
>
>
> On Thu, Jun 26, 2014 at 11:48 AM, doug <douglasrankine2001@xxxxxxxxxxx>
> wrote:
>
>> On 26/06/14 18:19, Aftermath wrote:
>> Dear Aftermath...or is it Bill?
>>
>> I thought this posting was absolutely brilliant. My congratulations to
>> you for bringing it to our attention and Mr. Wong wh did the initial
>> research. Why? Because...if you think on it, you have saved the NSA, the
>> CIA and the intelligence and security departments of the New York Police
>> Force huge amounts of money on expenditure of resources...and foreign
>> organisations such as the Chinese, the Israelis, the Russians and any ole
>> Tom, Dick and Harry of a security organisation a lot of time. Even the
>> United Nations security and intelligence services will be all agog at your
>> posting when they read it. I hope you are not claiming copyright on
>> it...;-) . At the click of a button, they can all find out who has been
>> doing what in New York....United Nations officials, Ambassadors, envoys,
>> consuls and all sorts of public and civil service officials, secret
>> service, intelligence services, even the private sector. All this metadata
>> available to link up with known associations and links...and all open
>> source too, who visited who and when, who was where and when, with a little
>> help from the smart mobile phone, the most valuable source of i.d. location
>> and contacts, you have provided the world, with a wealth of information.
>> Just think...the amount of money you have saved the world's intelligence
>> organisations. Absolutely f*cking brilliant....My congratulations...
>> ATB;-)
>> Dougie.
>> P.S. Who needs privacy and encryption when we have all this stuff about
>> the world leaders in our very own hands. It is all a question of pressing
>> the right button...
>>
>>
>> http://chriswhong.com/open-data/foil_nyc_taxi/
>>> and
>>> https://medium.com/@vijayp/f6bc289679a1
>>>
>>>
>>> from the second link...
>>>
>>> "
>>> Recently, thanks to a Freedom of Information request, Chris
>>> Whongreceived and made public a complete dump of historical trip and fare
>>> logs from NYC taxis. It’s pretty incredible: there are over 20GB of
>>> uncompressed data comprising more than 173 million individual trips. Each
>>> trip record includes the pickup and dropoff location and time, anonymized
>>> hack licence number and medallion number (i.e. the taxi’s unique id number,
>>> 3F38, in my photo above), and other metadata.
>>>
>>> These data are a veritable trove for people who love cities, transit,
>>> and data visualization. But there’s a big problem: the personally
>>> identifiable information (the driver’s licence number and taxi number)
>>> hasn’t been anonymized properly — what’s worse, it’s trivial to undo, and
>>> with other publicly available data, one can even figure out which person
>>> drove each trip. In the rest of this post, I’ll describe the structure of
>>> the data, what the person/people who released the data did wrong, how easy
>>> it is to deanonymize, and the lessons other agencies should learn from
>>> this. (And yes, I’ll also explain how rainbows fit in).
>>>
>>>
>>> The NYC taxi data consist of a number of CSV-files.....
>>> "
>>>
>>
>>
>>
>
>