Surveillance Design Final Project

Final Zine

Process & Concept

When you read a book that is full of imagery words, What ís going on inside your mind? What images do you have in your head when you reading?

Here is my project entitled “Mapping the Imagination.”

So, my project can be translated as “A Visual Practice of Human Imagination and Image Recognition from Text Redaction with Photo Data on Google Search.” So, this is my zine. For me, it was a really meaningful experience making a zine by my hand. It was almost first analog design project after DT and so much fun.

Along with the question that the machine can really recognize the images based on human imagination, I have been particularly interested in these areas of Human Imagination, Cognitive Psychology, Photo Vis,Computer Image Recognition Technology, Artificial Intelligence, Linguistics. So, this project was also an experiment to explore the realm of human cognition and artificial image recognition.

Inspired by some zines that Melanie showed us in the class, in terms of the design process and the overall concept, I did the similar way of text redaction.

As you can see, I cut each word in the text out in square boxes and there are two kinds of colored lines of blue and pink. The texts are from a famous fiction written by Marcel Proust – “In Search of Lost Time.”

Swan’s Way – Chapter 1 – In Search of Lost Time

The reason I chose this text for redacting is this fiction is well known as the most imaginative literature that is full of sensibility in human history, and this is one of my favorite fiction.

One thing that is also critical for this project is the process to search for photo data on Google.

As you know, there are a massive amount of big data including photo data, and the algorithm of image recognition by computer is unprecedentedly evolving based on those data.

but as you can see, in the process of choosing Images that match with words is based on human imagination and it’s an arbitral process.

So in my opinion, even though the computer can recognize text and match with the photo quickly, there is still a room for human imagination that affects the choice in visual search. Through this visual practice, I wanted to illuminate the fact that human visual system and imagination is still a rich source of information. And modern surveillance system cannot invade a person’s mental privacy yet because the cognitive processes of individuals are based on each person’s unique experience in the life and creative imagination even though the artificial intelligence like Google “Alphago” are evolving and even developing the creative process on its own.

After the final presentation, I really wanted to develop this project more by figuring out the correlations between the words and my cognition process of choosing those images.

So, for the further development, I made a diagram in a form of the semantic neural network with D3.js and JSON data to clearly visualize the invisible cognitive process of the arbitral image choice of mine and to figure the correlation between the words that I chose out. The chunking of words was exactly representing the images that I chose. So, It reveals the fact that the choice of images is not just a randomize, quick choice but a sophisticated cognitive process of human image recognition.

(below) JSON dataset that I reversely extract words from images.

Many studies in Machine learning and Artificial Intelligence are trying to extract visual classifiers from the human visual system. There is a field of Artificial imagination (AIm), also called Synthetic imagination or machine imagination which is defined as the artificial simulation of human imagination by general or special purpose computers or artificial neural networks.

Related to one of my final project concepts of self-tracking or self-surveillance (and also in the same domain with project 1 – 2), I found an interesting radio program that changes my viewpoint towards self-tracking via wearable devices for quantifying our daily life. This episode criticizes the rising culture of self-tracking in line with quantified self by highlighting many cases that a high level of awareness of daily life induces anxiety in some ways, even makes people feel like they are in prison. This discourse made me raise new questions: How safe are the self-tracking devices and app? How safe is your quantified self?

On this week’s episode, you’ll hear from Natasha Dow Schull, author of a forthcoming book called “Keeping Track,” and technology writer/early self-tracker and writer Paul Ford. Schull’s research has involved spending quite a bit of time in the aisles of Best Buy, listening in on the hopeful, aspirational purchases. However – as new research begins to bear out – respondents in the long run tend to fall in two camps: people who get turned off by the idea of self-tracking and need to be convinced of its value, or those who like the idea but want better technology. In both cases, the stalwarts of this billion-dollar industry are listening very, very closely to figure out what consumers really want from this trend.

We’re curious too, though for different reasons. We’ve spent the last few months asking a whole lot of people to speak to their experiences of quantifying themselves using technology. We wanted the story you can’t tell from the big tech conferences or even hanging out in the aisles of Best Buy. So we asked our audience to weigh in (figuratively, of course) on what makes for “useful” health technology – what different sorts of health hacking have really done to their health.

“Generation Like” is a documentary that shows the overall social media culture of teenagers today.

My undergrad major was PR & Advertisement, so I learned about new media marketing strategies and digital advertisement for cross-media platforms.

The documentary showed me the currents of the way the teens interact with the market by participating on social media.

By clicking the “Like” button and participating their favorite brands and entertainment’s campaign, they are not just being marketed but also being a part of marketing itself.

Also, the companies that provides marketing insights for their clients by using analysis tool for social media marketing are running the show. So the numerous numbers of “Like” by young users are dissected, analyzed and monetized in real time and advertisements companies are using the data.

For the concept of invasive looking, I would suggest a new desktop dashboard UI that shows our daily life data gathered by mobile devices and social networks.

Nowadays, everything is connected with our online identification and all the devices and social networks are gathering the daily data and activities in every seconds in case they can access to my account.

So, I imagine a dashboard that collects all the daily data and analyze them with algorithms and then visualize the data in a creative way to let us know the daily pattern at a glance.

To make a visual example of the dashboard, I gathered my actual daily data by a health tracking app, my scheduler and other social networks and then manipulated the data in Excel and Rstudio.

The prototype is a bit crude now, but I plan to develop this for a better version by finding some hidden patterns with the daily data that I’ve gathered. Also, I want to make an interactive version of it to connect those data in real time by using API and social network widgets.

For the context of surveillance design, It would be a surveillance system that is a kind of self-monitoring and self-regulation. The system can give some recommendations for an individual’s life based on the data. Also, by using those metadata, companies and social network services can figure a person’s hidden needs and behavioral pattern by analyzing the connectivity between each daily activity of an individual.

Design: develop concept for Invasive Viewing project to present in class next week.

I personally experienced in surveillance from above very recently.

Because I’m living in the school dorm, there are always CCTVs that are capturing my movement in the building. 2 days ago, I lost my backpack in the basement and I asked the dorm security guards for checking the recorded video to look into what was happening at that time. I stayed all night in the basement at that time, so I watched the surveillance video that took my every movement in the space from 11 pm at night to 8 am in the morning. It was such a shocking experience to realize the school is observing me all the time. Unfortunately, I couldn’t find my backpack, but It was thought-provoking experience of surveillance system within the public space.