Lifestream – Redesign history and bookmarking – Wei Zhou

MISSION

Welcome to lifestream!The goal is to find a cool way to organize history and bookmarking for the next-generation browser.Obviously those mock-ups are not good enough, that’s why I put it up here and I welcome all kinds of criticism.If you have any valuable suggestions, leave me a comment, I’ll visualize the idea for you, and I’ll improve my current prototype for you,I’m glad to support you to design your own fantastic web browser. Welcome to Mozilla’s Design 2.0 world.

DESIGN

History: it shows your location, weather and calendars(events), users could conduct a very visual search base on that.

SCENARIOS

Storyboard 1

Feature 1: bookmarks: Auto categorizing, sharing

Shot1: Alice works as a programmer in Google. She likes musics and movies. Today she finds an interesting music website: http://www.songbird.com.

Shot 2-5: She drags the website to the top of the browser, the bookmarks menu drops down, browser automatically categorizes this website to MUSIC, names it as Songbird, records the time as 3PM, June 28, in Mountain View Google Cafe 40, sunny, and tags it as P2P music downloading website, records she was chatting with her mom on skype and listening to Whitney Huston’s album at that time. Alice clicks SHARE, and record a short video to her friend Joy. A message is send to Joy’s friend bookmarks.

Shot 6 – 8: Joy is a toy designer in New York. She opens the web browser today and search:calendar. Her default Google calendar jumps out. She drags the website onto the browser background, then her browser is in sych with her calendar data now.

Feature 3: A cool way to find a web page

Shot 9 – 11: Joy is trying to find an important document her boss sent to her on IRC, but she lost the link .She switches the bookmark view to HISTORY view. She types “yesterday, office, around 2PM, Boss sent to me” in the awesome bar. The web browser finds the link for her.

Shot 12 – 14: Joy types “facebook related” in the awesome bar, suddenly all the related web pages zooms in(Those web pages are based on her browsing history, her friends browser history and her settings-she is a big music fan and she’d like to explore more music websites). A cluster of websites jump out, in this 3D space, Joy see big thumbnails like Myspace.com, twitter.com and youtube.com, etc., she also sees the other tiny thumbnails of music websites and toy-design websites which the browser recommend her to read.

Feature 5: Infinite scrolling

Joy is a curious person. She chooses all the networking website and clicks”Open all in one tab”, then all the websites are shown within one page, she presses her figure on the touch screen monitor and enter all the sites and scrolls down – she checks all the sites in 5 mins.

Feature 6: Bookmark a timeline

Joy bookmarks the set of time she visited all the networking websites, next time she could just search “8 PM favorites sites ” to do the same surfing.

DESIGN THINKING

1. The best UI is NO UI.

When I worked in Microsoft, my boss Dave Vronay said this sentence to me many times, he used that as an example that why so many people hate Microsoft products. He said if the UI component is dominant on the screen, it’s gonna distract people’s attention all the time and reduce their working efficiency.

Usability is not the only concern about the concept of “Zero UI“. We are not familiar with buttons and drop-down lists menus when we are still a baby. We tend to grap things, drop things, pick up things in a more natural way (Terry Winograd suggests that we interact with the world around us in three main ways: manipulation, locomotion, and conversation). When we focus on finishing a task, we never notice the way(interface) we approach the task. We are living on a ZERO-UI world, so Mark Weiser brings about the idea of ubiquitous computing.

Zero UI is not “sleek graphics + silver borders + minimum interactive transitions”. We need to work very hard to make the UI system “invisible”, or invisible enough. That requires a solid understanding of a “Digital Life“.That requires sensitivity and reflection, and most importantly, we need great imagination.

In the last past 50 years, computer invades our world truly like a monster. Sadly we built a bad habit – we got used to a monster-UI style. We are like a guy in jail hanging out only with guys, over 50 years, gradually lose the taste of real beauty.

ZERO-UI

Users never notice the UI when they are working on a task

Through the UI, they only see themselves, not the designer who design it.

They feel happy using it

They never think of how to use it, they just do

When they stop working, either just a glance, or a long-time gazing to the UI, they could be emotionally touched by the designer’s passion and love to her work.

They forget they are working in a computer, and think about nature when they are using it.

2. Using 3D in a 2D’s way.

The challenge of navigating through a 3D space is that people get lost too often. People who loves 3D are CG people – Don’t know why simply press the alt key and use the mouse navigating brings so much satisfaction to me) .Unfortunately normal users don’t. So far by reviewing a lot of 3D UI work, I found the big problems is: None of them are really making the 3D space”useful”.They are so many 3D for showing off purpose, 3D for thinking out of the box, 3D for getting people’s attention. So the question is: How do we benefit from the form(3D) to make the content(2D) more intuitive?

Combining 3D and 2D

Never move X, Y, Z at the same time.

Avoid forcing users to navigate through 3D space, if the purpose is to show off.

When users begin to get lost in 3D space, immediately switch back to 2D navigation.

3. No buttons.Only use buttons if you have to use it.

4. Touch. Stimulating 5 senses in a subtle way.

5. Devotion.Good design should make designers happy, too.

TODO

Special thanks to all the friends caring about Mozilla, I hope you could leave your real name for me to credit!

1. History can be sorted by time spent over a website

2. Tracking the amount of time you spend on certain web sites may help people realize they have a surfing addiction.

3. Content could be tagged and the relationship between tags could be represented in the form of a graph

4. Images could be searched by predominate color or range, or combination of colors

5. It can also describe what the bits of information look like, similar to files, they might have an icon associated with it. The environment can take all of this information and interpret it into objects in its world.

Great ideas, Wei. I particularly like the references you make to times, e.g. “8 pm favorite sites” and “around 2 pm”. Time is something that we rarely take advantage of as a landmark, yet as you demonstrate, it’s a very useful landmark, and also straightforward to implement at a technical level.

In Feature #6, though, is there a particular reason that Joy has to explicitly bookmark the set of time she visited the networking websites? Is there a way that Firefox can simply ‘enhance’ its history information to know what URLs were loaded in what tabs at any given time? That way Joy could get to this information even if she forgot to bookmark.

I don’t really understand why these sorts of interfaces are so in vogue among the design community these days. Though it might be “pretty” it is also resource intensive and tends to be in practice cumbersome to use. These sorts of features are really just gimmicks; do they really improve the utility of the web browser, fundamentally?

The real problem with user interfaces is pointless complexity which mostly follows from the inclusion of too many unnecessary features. What is really necessary is for programmers and designers to show some restraint and focus on building elegant and efficient programs that serve their purpose well, rather than including every possible little function and feature that more than five people, if that, thought they might like.

Today’s interfaces really are sufficient for accomplishing the task at hand. What really gets users is when you run slow, when you crash, when you blow away their profile at random, when you need to allocate a couple hundred megs of memory just to surf the web.

While I do realize that you’re just an art guy, I think the folks at Mozilla.org should focus on more pressing problems — like making Firefox 2 to Firefox 3 upgrades less prone to hosing user profiles, or tracking down the innumerable memory leaks. The interface is fine already.

Indeed I agree we haven’t solve a lot of pressing problems, but we cannot just solve problems, we need to experiment, take risks, fail and move on, right?

We could stay at the stage of satisfying a certain target of users, but if we can make everybody happy, why not have a try? So we have this basic interaction model: A fresh clean browser + many add-ons = a personalized browser.

its nice that everything is bookmarked, but …
how will the need to insert a “new” url destination be handled??
what about using the 3d interface to show links on the foremost page fan out behind the first page?

example .. http://www.cnn.com [there are lots of links on that page .. ]
can i indicate “politics” and all politics related links on that page fan out around/behind the main “www.cnn.com” page?

when i google something ..
can related links i am researching fan out where i can reach them [obviously they would have to ‘pre-load’ that i can view that when i indicate i am ready …

It would be nice to try this out to get a better idea for how it feels. Really great voice for the presentation. More from this person!

The thing I like best about this idea is how it automatically assigns all the meta-data to the bookmark, like the context and tags. If just this part works well, the whole idea should be successful.

Also really like the zoomed out spaces view. That seems like a nice way to see related bookmarks. I’d like to see more options on how they are grouped instead of just being based on time. Ideally, I could group them by related surfing ‘jobs.’

I’m curious in the browsing history view, what are the vertical columns? All the sites visited on the same day?

Dragging around presentation views and sharing them looks fantastic. Would love to try this out.

I agree with the basic interaction model you described…”A fresh clean browser + many add-ons = a personalized browser.” my current firefox setup is rather plain…but i have a few addons i wouldn’t live without. so that model works.

as far as this book marking scheme…
I love {LOVE} the concept of being able to query you history according to time, date, location, activity, etc. that’s brilliant. that’s the way i remember stuff anyhow. bookmarks exist to serve two purposes, 1, remember the thirteen odd weather sites that i like for me (all i have to remember is that i have a folder called WEATHER) 2, help me find individual sites i liked that didn’t have enough “friends” to warrant a meaningful folder/tag/whatever.

I’m always a fan of synching. make sure that your final design has a way to synch across the web…and maybe even to a portable app.

like a said, i like the relational search, but i’m also paranoid of others having access to such a pervasive record of my daily life. i’ll never use this unless you convince me that my data and activities are SECURE.

if the software is generally going to tag each site automatically, all you need is one button/flag/star to mark that page. maybe after hitting that button a popup window would show you what tags were assigned and let you force other tags.

i don’t want my screen cluttered with an inch high banner all the time. when i want to access my bookmarks i’ll click a bookmark button(a list will appear[without obscuring the screen] and that would be where your colorful-folders-at-the-top-of-screen would be. each representing one of my most used/favorite tag categories. i would put the main “show bookmark tags” button on top left of screen. push it and the tag lit would cascade out to the right across the top.

I like accessing my bookmarks using categories and sub categories. ie. sports/football/equipment/helmets/how it’s made.com.
even if i don’t tag site myself, i want that functionality.

i like the idea in picture 21, the history view. although this should be augmented with an in depth text based query function.

the history view could be called lifestream since it could show the music you were listening to, where you were, what computer you were using, the weather, etc etc maybe even tracking things like ambient light.

if the thing turns into a memory/speed hog, forget it. no one will use it.

if your just doing bookmarking, this would be fine. …but it would be great to integrate this with a new wave browser.

Oh, and have a web accessible interface so that when i’m at another computer i can just open that site and it will integrate with the browser i’m using as if it were actually installed!!! DO THIS!

I think your one of the most creative young person I’ ve come across in a long time
The UI , you designed is great , yet the on e element that i feel is lacking in browsers and the way I interact with them is that they book Mark a single page , with out giving you the option of book marking the home page .
It’s a small thing , but one that would be of better use to me , because I book mark sites that I visist to read something and the content is dynamic , when I go back I directed
to old content .

thank you so much for this long feedback! I’m an Asian and my English is bad. My appreciation of your feedbacks cannot be expressed by English words!

Security is very important, I agree. I know there would be someone ask this question, and my answer is, yes, security for a Mozilla is far more important then anything else. In my graduate research, I consider the software functions as a “virtual being”, it records what you are doing, in order to get to know who you are, what are you trying to do. It has nothing to do with artificial intelligence, it’s all about human-centered design. I’m recently thinking about making the web browser into two modes: private mode and public mode. So only within the private mode you would be able to use context awareness and relational search feature.

I’m a big fan of synching, too. In the future, we may be able to drag firefox browser across different kinds of portable apps in a really easy way. It’s a very interesting topic to explore.

“maybe after hitting that button a pop-up window would show you what tags were assigned and let you force other tags.”I think this feature is included in my browser concept but unfortunately I don’t have to explain it in a 3 mins video.

You won’t have your screen cluttered with an inch banner all the time, because it automatically hides itself when you begin browsing. I like the idea of “show bookmark tags”, it’s very useful. But I don’t know, sometimes when people add a feature, they add a button. It’s all about adding rather than reducing. It’s hard to simplify the whole UI without losing any features, but still create a clean workflow.

It’s a great idea to have categories and sub categories, and I just realize it might be a great way of exploring 3D UI design, you know, make the Z axis useful. Hmmm…

Lifestream! What a wonderful name! You are a genius! Ambient light…such a wonderful idea! It is a very functional as well as a emotional feature! I’ll credit your name when I add this feature to my prototype.

“DO THIS “- check about weave project in Mozilla Labs, we are doing it now. 🙂

William:
“I think your one of the most creative young person I’ ve come across in a long time” – thank you so much for the encouragement! 🙂

“in browsers and the way I interact with them is that they book Mark a single page , with out giving you the option of book marking the home page .”
Interesting. To solve this problem, let me give myself examples.

Sometimes you do need to bookmark a single page – like your receipt. – now we consider a web page as an “INDEPENDENT FILE”.
Sometimes your goal is to remember the homepage URL – we consider a web page as an URL string.
Sometimes you bookmark this web page only because you like a certain content in it, like a video, and you’ll visit that later to share with friends. – we consider a web page as a container of DATA.
…
Then should we allow people to bookmark a page as “the whole page as a file, a url, or part of the content”? Very interesting.

“I’d like to see more options on how they are grouped instead of just being based on time. Ideally, I could group them by related surfing ‘jobs.’”
YES. You would be able to do it. We decide to limit the whole concept to a less-than-5-mins video, so we cut a lot of useful features, one of them are the feature of allowing people reorganize their timeline objects by breaking the idea of “timeline”. The objects(website thumbnails) displayed on the time-line are not really timeline based, but content-based, meaning, it’s a non-linear timeline.

Wow, I feel like I’m lacking the ability of explaining it using english words. I’ll try to update my prototype blog by presenting how you can manipulate with this awesome non-linear timeline, now we have a name for it:”life stream”. 🙂

“how will the need to insert a “new” url destination be handled??”
Click anywhere of the blue background to input a new URL string. Ideally people would input it at the top of the screen.

“what about using the 3d interface to show links on the foremost page fan out behind the first page?”
Awesome! Another big headache is to design the interaction model not web thumbnails but pure URLs, because sometimes people just hate thumbnails, and we should be able to make those people happy, too. I’m thinking making a pure URL web interaction mock-up. All the URLs are colorful and animated in a meaningful way. 🙂

“example .. http://www.cnn.com [there are lots of links on that page .. ]
can i indicate “politics” and all politics related links on that page fan out around/behind the main “www.cnn.com” page?”
That would be really cool. Thank you so much for bring the idea. And using 3d space to do it is just fantastic! I’ll try to visualize your idea when I’m done with my firefox phone design project.

I have the question all the time in my mind”How do we really making the 3d space useful in the 3d UI?”And your suggestion sort of opening the door of solving them. Yeah a single webpage is a 2d object, but the content in the page is multi-dimentional!(Wei is thinking something super cool now). Think about a 3d-rendered Google logo pops out of the 2d google page and project shadow on it, think about the searching words floating on top of a web page and move around as you interact with them…and a video of a website jumps out of that website and begin to dance with your mouse…I have no idea what I’m talking about now, my english is BAD.

I love the ideas you’re working on here. The history view in particular is awesome.

Personally, I’d love to see you mix in more of the enso/command line concept. I’d much rather type “file under music” than drag it (even better if the suggestion system defaults to the appropriate category so you can type “file” and see that it’s going to file it under music for you). Not suggesting you replace one with the other as much as you add it as an additional way to work with the system.

My one criticism is it seems you’ve traded text labels for images wholesale in some of the views (as you mentioned above). I don’t like the idea of dumping one for the other though (you said you were considering an pure URL interaction mockup). What about having a column as sort of a focal point where that column had text but none of the others did? You could then slide the label column over the view or slide the view under the label column. It would give you a tradeoff where you kept labels without giving up too much horizontal space. This might be a further zoom level … i.e. normally users get the view you’ve shown but if they want lablels they roll forward one more notch.

Rick:
I really appreciate your feedback! I and Aza talked a lot about how to combine enso’s idea into the browser. Mozilla Labs is working very hard on making it real. For now in our prototype, you would be able to type”download, email, share, print” to perform a task flow like “download a video-> email the link to yourself->share the link to your friends->print the file out. ” I believe you will feel as excited as I do. 🙂

Yes I agree. My concept video for now is targeting for only non-power users.”Dumping one for the other” is a very bad decision – I actually forget putting labels on my thumbnails(I hope you could forgive me if you know I only had two weeks on this project and I was lacking of sleeping all the time).

Your idea of the “labels” are cool, and it’s a efficient way of using 3d space UI. I spent a lot of time today on reviewing people’s feedback about my concept, I found a guy’s idea about “layers” might solve the problem. I’m not sure I fully understand your column, it would be great if you sketch out for me, I already tried my best and read your comment 14 times, English is not my first language.

I want to preface this by saying there’s no way I’m the first person to have thought of this but I’ve never seen anyone talk about doing it this way in the browser itself (admittedly though I haven’t looked all that much).

Instead of thinking of history and bookmarks, have you considered thinking of it as data and views? I.e. you have one giant list of URLs. Each has the last date visited attached to it (i.e. the history data). Each could also have tags, context, etc attached to it (i.e. the bookmark data). Every URL appears in the list once and has as much data attached to it as needed (so the same entry could have both the history data and the bookmark data). Once every URL is in the same data format, you can display any URL using any view.

You could make views like “view by date”, “view alphabetically”, “view by tag”, etc. To show a specific set of information you select URLs from the list and display them in one of the views. For a traditional history list you would select every URL in the system and display them in the “view by date” view sorted by the “last visited date”. (by select I’m using it in the query sense, not in the sense of the user actually selecting things). You could just as easily show your bookmarks by the date they were added if you wanted to.

The advantage of a system like this is Firefox extensions could be created to add their own data to URLs in the list or to create new views. As an example, a wikipedia extension could add historical dates to URLs. The US declaration of Independence would get a date of July 4th, 1776 and if you scrolled a “view by date” view back there you’d see it at that date along with the link to the wikipedia article. If you scrolled forward a bit to July 21st you’d see Mozart’s Haffner Serenade was performed in Salzburg. You could even couple this with opt-in URL feeds to generate all kinds of views of the web in general right in the browser.

Honestly, I have no idea how this affects your UI. I think instead of having separate views you’d have more of a single screen where you choose which way you want to view the data and what data you want to view. Mostly, it’s just a different way to think about it.

Kudos on replying to those who posted, Doing that makes us feel like we have a part in creating a useful tool. I’ll check back here from time to time to see how you are getting along. I for one would be happy to get an email when this concept goes into beta.

I looked up the “weave” project. That will be amazing! i work on many computers and look forward to the day where i can access my bookmarks etc from any computer 🙂

I agree with your response to William, that you could have the option to bookmark a spesific page to keep track of the content (ie. a video,pic, or blog post) OR you could bookmark the blog homepage. I think it will be important to make this selection easy to remember…using words or icons. along the same lines it would be nice to have an easy way of saving online content. for example, say you go to a blog and you like one section of text with a photo. it would be really nice if you could select an area (like using clipmarks) and “bookmark” that content (as an object)and simultaneously have the browser save that content to a online storage area (like the google cashe). that way if the website goes down, the content is still saved, and all that happens without user interaction.

Rick makes a good point. storing all url’s as database objects allows almost limitless ways of sorting. I’m not sure if this is standard in current browsers, but it seems like a good idea. I think opening up the possibility of users writing extensions and addons that can directly modify the url database would let people organize in any way they were comfortable. and that’s always a good thing.

I was interested in it when taking a glance of the images. But I have to say, after taking a close look at the idea, it’s just an combination of the “Apple looking” and “old concept of bookmarks”, which even makes it crumbsome.

I love it. I can’t wait for the future. The idea of zero ui blows my mind. I think about that when now when I use Quicksilver on my mac. I never really think about opening applications. I just do. These ideas are brilliant!

Thank you for your support! I’ll definitely keep in touch with you as the project goes to ship …what is your email? I already add my email address to the “About me”session: weizhou@andrew.cmu.edu

I agree with your comments. I like Rick’s idea, too. People already take URL as granted, but never think abut further manipulation. For a browser’s UI, you may notice the URL bar takes the most space, but it’s actually not functional at all(except when people needs to copy and paste the URL).

I use Clipmarks quite often, unfortunately it’s not compatible with “Stumble” well.As the concept of Semantics web goes, I believe in the future, people could both view the web in both “Micro” and “Macro” ways, which requires a “deconstruction” of the web content.

Wei, this is simply, absolutely, definitely AMAZING !
You went from revolutionary ideas to a beautiful demo of a new (no) UI.
I want it !!!
Serious question : when could we expect to see that for real ?

I love your idea..and inspired!!! Keep on good work!!!
I have no drawings nothing to share but an idea.
My problem is that my bookmarks “grow” by time more and more nice sites coming and many to come…i liked the part of a time line on bookmarks but i want something more..something like a search engine on bookmarks based on a small database!,cause time makes me sometimes to get confused.Finish with this short prologue.
1.Adding bookmarks.Lets think i am looking for some holiday…about a place and found some nice travel sites that point me to that place.I want to be able highlight the text on page “travel to place” right click and insert to bookmark folder (or category you can call) “travel to place” automatically.Soon or later there going to be many categories.
2.Find bookmarks.Same way as before in adding.. highlight text-right click but now i can choose even words like “travel to” cause i liked the last travel agencies i used but for different destination.
3.Sure must be a way to manage bookmarks to subcategories etc.. leaving it to your fantasy(with simple drag and drop i hope).
4.Search like CTL-F in bookmarks too!!!!
We try to serve our needs here and hopefully create a handy-useful interface.Firefox is a “pagereader”and that will remain what else we might want more? (..we can leave it on extensions)
Saying again i liked the whole project and hopping soon on 3d.. but dont forget that really good ideas start from 2d or even script..

Someone should take a look at better utilizing dual screens. I’m a CG/ Digital artist. Every other artist and major companies use set ups with multiple screens.

I would like to see features like arranging program windows for me based on some sort of auto memory or in preset configurations like viewports in 3d software. It’s a pain because I usually have 3-6 programs open when I work in CG, such as: a list of keyframes etc. in notepad or microsoft word, my 3d package, photoshop, a game engine or highpoly modeling software, reference images or maps in the windows preview, internet, multiple folders, music player, misc programs like a top level screen drawing program or video capture.

When deadlines are looming over our heads, we need every second we can get. There has to be some way to make it easier.

I’m a programmer with an eye for UI design myself, and I have to say your work is very inspiring. Programmers make the language worth learning, and your work does wonders for actionscript. It really provides a challenge for non-web based developers (like myself) to bring the same level of creativity in our interface or to design our own tools to compete with the amazing work coming out of the web design industry. Thanks for the glimpse into the future, good luck in all your projects !!

What i do not understood is in truth how you are not actually a lot more neatly-preferred than you might be right now. You’re so intelligent. You already know thus considerably with regards to this topic, made me personally believe it from a lot of various angles. Its like men and women aren’t involved until it’s something to accomplish with Girl gaga! Your individual stuffs outstanding. Always take care of it up!

Hey there would you mind letting me know which web host you’re using? I’ve loaded your blog in 3 completely different web browsers and I must say this blog loads a lot quicker then most. Can you recommend a good internet hosting provider at a fair price? Thank you, I appreciate it!

I didn’t really believe that you could just “look it up on the internet” and find out how to make this awful stuff but I looked and there it was. I so appreciate your humor in the middle of a pretty depressing browse around the net. Thanks

a lot more neatly-preferred than you might be right now. You’re so intelligent. You already know thus considerably with regards to this topic, made me personally believe it from a lot of various angles

[…] with Mozilla Labs is available on Adaptive Path, LLC website. Other inaugural concepts are Bookmarking and History Concept (a study of interaction nature between user and browser during the bookmarks and history management […]

[…] inconvenient, it’s because I thoug to another way to classify. Wei Zhou, in a comment on her LifeStream article, propose to classify bookmark in 3 part : DATA, URL and INDEPENDANT FILES. I have already thought […]

[…] they could focus on making their browsers have more features that would satisfy the end user, like bookmark and history visualizations (just make it easier to find a particular website that I visited!), and making data movement […]

[…] However I am yet to find such a system that I like. The concept in my mind is something like the Lifestream design that Wei Zhou blogged about. An interface where time is on the horizontal axis, where you could change the scale and location […]