Deep Fictions

Deep Fictions utilizes artificial intelligence and natural language processing to generate portraits of characters from famous novels based on their textual descriptions. The project speculates whether our biased imaginations can be modeled, and explores the potential blind spots this process might help us reveal. The examples shown here are of Dracula from Bram Stoker’s eponymous novel, and Elizabeth Bennet from Jane Austen’s Pride and Prejudice.

Our culture, environments, and biases all help to shape the images we see in our mind’s eye as we read texts — of novel protagonists, for example. We need look no further than Hollywood’s whitewashing of cinema to see often problematic depictions of beloved characters. As AI improves, could generative storytelling help to stretch our imaginations of who we envision in popular narratives?

Through this research, I wanted to explore whether it’s possible to diversify homogenous datasets through generative neural network algorithms, and to examine how such algorithms might reflect on our own imaginations of characters from popular culture.

Wikipedia Was Here

Wikipedia Was Here reveals the location and geographical context of anonymous edits to specific Wikipedia entries, as they occur in real-time. The site geocodes the IP addresses of new Wikipedia edits and shows corresponding visual imagery on Google Maps and Google Street View.

Wikipedia entries are, on the surface, facially neutral. But in the name of crowdsourcing and “truth,” certain edits aggregate over time to semantically erase and reshape important popular narratives. Further, anonymous edits obfuscate context, authorship, and power, often marginalizing voices from minority communities.

When are anonymous editors leaking crucial facts? Drawing upon local knowledge? Making assertions without evidence? In a “post-truth” era, Wikipedia Was Here aims to make the production of popular knowledge—and the underlying power relations—more visible, helping us to visualize the cultural and infrastructural architectural contexts, and perhaps even a hint of the political climate, in which entries are edited and rewritten.

In March 2018, YouTube announced that it would add information from Wikipedia to videos about conspiracies. Given how Wikipedia entries can be anonymously manipulated, this project also questions whether such moves are sufficient in combating fake news. (View the project: https://wikihere.justin.work/)

When editing a Wikipedia article anonymously (without an account) the editor's IP address is used as their username. According to Wikipedia, VPN's and Tor are mostly blocked when anonymously editing articles.

From Wikipedia:"Editing Wikipedia with an IP address as your identifier is often less anonymous than editing with a normal account.... Note that editing Wikipedia through Virtual Private Networks anonymizing proxy servers or Tor is, at a minimum, controversial, and users editing via these proxies are most of the time blocked from editing, until you disconnect from anonymity networks due to problems of vandalism on anonymous networks such as tor, proxies, and VPNs."

More information about anonymous Wikipedia edits and IP addresses can be found here.

An anonymous edit to Donald Trump’s Wikipedia entry the day after he was elected President.

Hyper-Blocks

Hyper-Blocks uses open civic data to algorithmically reveal recently constructed or altered city blocks, with newly concentrated ownership of property. These city blocks go hand in hand with what some assert is a new form of gentrification, “hyper-gentrification” — one marked by corporations and governments, rather than individual residents, remaking the city. The heft of the new developments highlighted in the project, transcending traditional notions of single buildings, prompted me to call them “hyper-blocks.” These developments are often owned by a single developer, or are part of a larger development that spans multiple blocks.

These hyper-blocks have fundamentally altered the fabric of Brooklyn neighborhoods and the built environment. Viewed from above, these developments mimic “superblock” planning revered in the mid-20th century, whereby large blocks form micro-neighborhoods. Such micro-neighborhoods provide residents with the recreational areas they needed to socialize amongst themselves, but also cut them off from the “outside” world.

Hyper-Blocks is an extension of a project I began in 2013, Vacated, that visualized how zoning changes during former Mayor Michael Bloomberg’s administration transformed New York, from a street-level perspective. Vacatedused civic data to mine cached Google Street View imagery, revealing photographs of urban change in New York City that had remained hidden in plain sight.

ProcessHyper-Blocks draws upon the NYC PLUTO dataset to group all property lots in Brooklyn into city blocks, which serve as the primary unit of analysis. It then searches for city blocks with at least one new building (constructed in the past 10 years, since Bloomberg's zoning changes). Finally, it aggregates city blocks with the highest numbers of residential units, but the fewest number of individually owned lots. This yields blocks that are often owned by a single developer, or are part of a larger development that spans multiple blocks.

Year: 2016

WSSID

WSSID (Weather Service Set Identifier) is a hacked wireless router that collects and broadcasts live weather data from its local environment in the form of dynamically changing SSID names. The project examines the use of wireless networks on the environment and how we are always seemingly in range of these signals without explicitly asking to be. Operating similarly to the way in which weather is beyond our direct control, “WSSID” shows us how permeating these networks have become by relaying direct environmental data about the locations in which they are housed.

The Book of Answers

In 1973, while on his deathbed, Pablo Neruda penned 316 questions as 74 poems posthumously published in The Book of Questions.

The Book of Answers is an anthology of responses to each of Neruda’s poems. These poems were compiled using Amazon’s Mechanical Turk (MTurk), a utility that allows anyone to submit jobs to a global online workforce. The project attempts to (re)humanize this de facto dehumanized anonymous workforce by engaging workers with Neruda’s seemingly unanswerable questions.

Each of Neruda’s questions were embedded in a different menial survey answered by online workers. For example, after answering whether they strongly agree, agree, disagree, or strongly disagree with a statement about climate change, a worker is asked “What do they call the sadness of a solitary sheep?”

What happens when these two very different practices of labor and poetics collide, in an online marketplace? When, in the midst of menial labor, do we find poetry?

Year: 2013-2018

Vacated

Vacated reverse engineers Google Street View to highlight the changing landscape of various neighborhoods throughout Manhattan and Brooklyn. The project finds cached, historic images of New York City hiding in plain sight on Google Street View. These images are algorithmically extracted by merging the Department of City Planning's PLUTO dataset with Google Street View, and searching for all properties that have been altered or constructed since Google Street View began (2007.) Vacated was conceptualized, built, and released 8 months before Google publicly released its Time Machine feature, which allows users to view historic images for given locations.

Vacated mines and combines different datasets on vacant lots to present a sort of physical façade of gentrification, one that immediately prompts questions by virtue of its incompleteness: “Vacated by whom? Why? How long had they been there? And who’s replacing them?” Are all these changes instances of gentrification, or just some? While we usually think of gentrification in terms of what is new or has been displaced, Vacated highlights the momentary absence of such buildings, either because they’ve been demolished or have not yet been built. All images depicted in the project are both temporal and ephemeral, since they draw upon image caches that will eventually be replaced.

Ultimately, Vacated is a walking tour of the changing urban landscape during the Bloomberg administration— it depicts some obvious examples of dramatic change, but in the end, it's up to the viewer to decide whether this change represents widespread gentrification.

Process

This project began by examining whether temporal differences in Google Street View's cache could be used as a device to narrate rapidly changing urban landscapes. Since Google updates major roads on Street View more frequently, buildings located along intersections often contain images from two different time periods. This temporal gap can be large enough to span the entire construction of a new building.

To find these some of these gaps, I used the NYC Department of City Planning PLUTO dataset to extract addresses that had been built or altered since 2007. I then scraped Google Street View images for these locations to find cached, historical images.

During my research, I found that Google Street View cars often capture intersecting streets at different time periods. These time differences range from a few months to more than a year. If you visit these intersections in Google Street View and cross the intersection, the time period will change, revealing before and after imagery for the block. The PLUTO dataset contains a "Lot Type" field for each entry, which I used to find only properties that are located on intersections and were built or altered since 2007. Using this filtered dataset, I created a new script that scraped imagery for 2 different positions at each intersection, capturing before and after images for blocks en masse.

In April 2014 Google released a large portion of historical Street View image data through a tool called "Time Machine" tool. This historical imagery had previously remained unavailable to (or hidden from) the public for over seven years.

The imagery collected for Vacated was only accessible by data mining and cross-referencing the Google Street View and New York City's PLUTO datasets to find cached imagery along street intersections, where two separate cars had converged at different time periods. The new Time Machine tool allows all of us to more easily reflect upon historical images of any building or lot, on intersections or not.

Dumpster Drive

Dumpster Drive is a file-sharing application that recycles digital files. Using dumpster diving as a model for recirculating unwanted objects, Dumpster Drive allows others to dig through files that you delete on your computer in a passive file-sharing network. Instead of simply erasing data from your computer, the software allows users to extend the lifecycle of their unwanted files and pass them on to others.

Waste.fm

Waste.fm is a radio station that broadcasts discarded audio files from around the world. Users take the place of DJs, contributing to the radio station whenever they delete files on their computers using the Waste.fm desktop application. Waste.fm uploads these files to the radio station and broadcasts them both on the project website and desktop application. The radio broadcast plays every deleted audio file once, and then permanently deletes it from the archive.

Numerous social music services use collective data to recommend and serve us songs based on our listening histories and online friends, leaving us at the mercy of what algorithms such cloud-based services use to determine our artistic tastes. Whether these algorithms are diagnostic or prognostic, they have changed how many of us experience music today. What happens to the musicians who are outliers of established genres? Are there more serendipitous ways for us to find music that skirts the periphery of our predicted repertoires?

Waste.fm lays the foundation of a digital commons for data that we typically share only selectively, or keep permanently private. It allows listeners to simultaneously experience typically unshared content in a collective environment. In doing so, it hopefully throws into sharper relief the barriers we typically build between our online and offline lives, the aspects we choose to share on major social channels, and the personas that we curate.

Dark Side of the Prism

Dark Side of the Prism is a Firefox Add-on that provides a soundtrack for our surveilled internet meanderings.

The public recently learned that the US National Security Agency's on-going internet surveillance program, Prism, collects data from users of major websites.

Many of us already know that any data we might share-- not just Facebook posts, but our search and click pathways and histories-- could be compromised, but we do so anyway. We have normalized this ubiquitous surveillance.

Dark Side of the Prismuses Pink Floyd's aural prism (Dark Side of the Moon) as a playlist to the NSA's tracking efforts, serving as an auditory reminder of how our online activities are surveilled. What hypochondriac questions do you Google in the middle of the night; who do you cyberstalk? Consider those missives the lyrical component to our soundtrack.

We Read, We Tweet

We Read, We Tweet geographically visualizes the dissemination of New York Times articles through Twitter. Each line connects the location of a tweet to the contextual location of the New York Times article it referenced. The lines are generated in a sequence based on the time in which a tweet occurs. The project explores digital news distribution in a temporal and spatial context through the social space of Twitter.

Snoozy the Sloth

Snoozy the Sloth is a breathing plush toy. He sleeps while clinging onto a user, allowing them to feel both the contraction and expansion of his chest, as well exhaling air from his mouth. The main concept behind snoozy is to create an intimate, yet passive, toy interaction that relaxes and comforts a user, through the tactile experience of steady breathing patterns.

Bit by Bit

BbB (Bit by Bit) s a cloud shrinking service create by Justin Blinder and Benjamin Gaulon. The project started from a common interested for digital waste and data recycling. During Art Hack Day at the 2014 Transmediale Festival in Berlin, BbB was be presented as a network performance and installation.

BbB displays the latest pastebin uploads, while at the same time, requesting for the content to be removed by sending automatic cease and desist emails. Each file displayed is visible for the last time at Transmediale 2014 and projected onto light sensitive surface leaving an afterglow for a short amount of time. BbB reduces the data accumulated on the internet, bit by bit. The project highlights how many cloud-storage services compliance with copyright laws create a facçade of persistence for our digital data.

WiFi Spotting

Using NYC.gov wireless hotspot data, WiFi Spotting topographically visualizes Wi-Fi saturation in the metropolis. Areas with higher saturation of access points form the peaks of these mountainous terrains, thus lending physicality to the usually ephemeral in our constantly changing cityscape. “WiFi Spotting” highlights how our immediate environments are saturated by constant signals, and it aims to materialize the underlying social contracts hidden within our ubiquitous Hertzian noise.

Ethnobotanical Station

Ethnobotanical Station (E.B.) is a mobile module that draws upon the diverse lineage of knowledge to study the complex relations between plants and humans. It brings in the question our faith in modern quantitative science as compared to the long tradition of qualitative indigenous knowledge through an inventory of distinctive tools, exemplary specimen and mappings that explore new ways to relate to the plant life around us. A combination of mythology and science fiction combined with qualitative science is used to create an experimental framework that regenerates traditional knowledge. Hands-on workshops and visual display are the vehicle for exploration and sharing new configurations of knowledge. Just like the intricate mechanisms for seed dispersal, E.B. moves freely to collect and disperse knowledge freely."

StockBank

StockBank is a networked piggy bank with the end goal of collecting enough money to buy one share of the three technology stocks of Google, Apple, and Facebook. As someone drops a coin into the bank, the display shows the amount of the selected stock share price decreasing until it levels off at zero, which means that there is enough money inside the bank to buy one share. The bank then automatically connects to an online broker, purchases the share, and blinks the pig. When the bank is emptied, the counter resets back to another share price and begins the process over again.

The project exists as an open source, networked object that prompts users to stay connected to the current stock valuation of technology corporations and see how they change and fluctuate over time. It reinvents the classic piggy bank by instead of counting money for saving, it connects in real-time to actual stock prices and gives the user an estimate on the amount of money they need to purchase a share of the specified stock.

GhostCoder

The GhostCoder uses viral dynamics as an archiving platform to ensure important data stays within the public domain.

The GhostCoder is an encryption utility designed for the sonic weapons research collective AUDiNT, comprised of Steve Goodman (Kode9), Toby Heys, and Jon Cohrs. The tool embeds audio files within FLAC music files using spectral analysis, encoding this new audio beyond a listener's hearing range. This hidden information piggybacks on top of popular music files and torrents without being noticed, thus creating a networked, distributed archive of secret information. The tool was originally built to proliferate the spread sonic warfare research, however it allows users to secretly embed any audio file. The hidden data can be decoded using the GhostCoder to hear the embedded audio.

The GhostCoder can be downloaded here and the source code is available on GitHub.

Role: Tech LeadYear: 2011

Baking with Augmented Reality

Baking with Augmented Reality is my first project in a series that attempts to augment menial and common tasks through AR, serving as a response to how AR is being used to ostensibly enhance mediocre products. This project consisted of a cake with a frosted fiducial marker on top. Building a custom application using OpenFrameworks, I tracked the marker and superimposed a decoration on the cake.

In the end, I went for a modest decoration: Keanu Reeves riding a unicorn, with rainbows and flying dolphins.

Audiowire

Audiowire is an interactive visualization that depicts associations between different musicians. The project explores how online music recommendations services are constructed, and identifies artists that fall might outside of our curated profiles. The project uses the Last.fm API.

Year: 2008

ReThink

ReThink is a FireFox add-on that seamlessly integrates environmental footprint data into corporate websites. The software aggregates environmental accountability scores and statistics, and injects this data into the website's existing HTML markup.