A very interesting observation during the talk was that Twitter started up with a CMS model and that they gradually moved towards a messaging model. I’ve seen this in a few applications so far, including a casino system, where the messaging model seems to fit best an application intended to power massive community of online users, it seems regardless of what the application actually does business wise. Applications start out completely different, but then more and more functionality gets bolted on top of user messaging capabilities that the whole architecture on the end gets refactored to utilise the messaging channels as the core information transport. With Twitter, I’d expect this to be more obvious from the start as it was intended to help people notify each other.

The interesting thing, however, was that all the upgrades were done live, without shutting down the system. The changes were always introduced to one node, then regression issues were sorted out, and then the software would be rolled out to the whole cluster. They went as far as building a whole messaging system based on memcached APIs in order to be able to slot in such changes.

a write-through vector cache of primary tweet keys with 99% hit ratio, a write-through row cache for tweets and users with 95% hit rate and a read-through fragment cache with rendered versions of different tweets for different clients with 95% hit rate. All these caches are based on memcached.

this change allowed them to increase the web server performance from 3.32 requests per second without caching to 139.03 requests per second. Weaver said that API services work about four times faster than the web, which means that the API performance is roughly 550 requests/s [my calculation, not given during the talk].

Web is only 10-20% of the traffic, the rest is through API services

Web servers are still 50% of the cluster.

Regular incoming traffic peaks are around 80 tweets per second. I expected this to be a lot more.

before the upgrades their web servers shipped only 3.32 requests per second!

For each tweet, message gets inserted for each user which follows a tweet. In average, a user has 120 followers so this comes to about 9600 messages/s at peak times

Message servers run on three nodes. They decided to write their own messaging software in order to make the protocol memcached-like, and did not evaluate other available solutions.

During Obama’s inauguration, they peaked at about 350 tweets per second for around five minutes.

They had a ton of problems with garbage collection but strangelly haven’t looked into JRockit RT or something similar that has predictable GC. Twitter JVM middleware runs on the SUN JVM

Most of the tools used by Twitter are open source. The stack is made up of Rails for the front side, C, Scala and Java for the middle business layer, and MySQL for storing data. Everything is kept in RAM and the database is just a backup.

I write between ten and a hundred notes each day. Sometimes I add to an existing note or document. I have trouble working with notes, documents, ideas and streams of thought.
Do you recognize this? How do you handle it?<p>Do you keep notes in one place? How do you accomplish that? How do you handle physical notes and non-physical notes? Do you try to gather all notes online? Where and how?

I don’t.

Like you, I probably have about a thousand different ideas and streams of thought during the course of a day. Unlike you, I don’t write any of them down.

I do this because there is no way I’d get anything done if I didn’t. I don’t mind forgetting some key insight, because if it was important or relevant enough, it’ll translate into sometime I will write down when the time is right. If not, it’s probably something that I shouldn’t bother wasting my time with. (That’s not to suggest it isn’t valuable).

You only have so many hours in the day to work on so many things. People like us need to ignore our own brain 80% of the time to be productive. It’s a curse really.

Org-mode is nice for two reasons: first, it gives you a way to organize projects and sub-projects and meta-projects. Second, it’s a reality check — if you gave yourself a task that in no way advances any goals you’ve already stated, org-mode’s organization scheme means that you end up either creating a new goal or admitting that a new project isn’t worth your time.

I wasn’t happy with any of the solutions out there so we built our own.

We treat notes as a stream and you categorize notes using hashcodes.

We have an iPhone and Android client so you can easily include pictures in your notes.

Thanks, we’ve looked at those before and they still seems a bit too complex. I do like their audio-text bridge.

We are building something even simpler, a personal syslog synced across phone/web/command line.

Fundamentally we view notes as a stream like a log file or Twitter.

You will have one place to easily and quickly dump everything, then be able to grep out the relevant information, and then late bind the decision on what to do with that information.

I have 744 “Drafts” to myself in GMail.

Ditto. But I manage (somehow) to loosely categorize them: there is an (ever expanding) message with Music, Movies, Books and other media that I want to look at.

EVERNOTE!

I can’t say enough good about this program. i use it everywhere (mac, web, macbook, iphone). it’s certainly not perfect (auto indent?, the ability to copy check boxes) but it is certainly good. I use a moleskine too, but for searching and loading up data, and keeping task lists, with pictures! Evernote has made me very happy.

this question was asked awhile back on HN and that is how I found out about. I’ve tried various things in the past, but evernote has worked great for me. evernote.com.

I second or third that. Great on the mobile phone for uploading quick notes and pictures.

Evernote is to my memos what dropbox is to my work in progress files.

Onenote is great indeed. Possibly underrated because it is a MS product, but it is great for note taking and syncing those notes to pda or mobile phone.

Warning, Mac-centric answer:

For task lists, I tried a bunch of things at one point, and only one that stuck was Taskpaper (http://hogbaysoftware.com/products/taskpaper). It’s so simple that I actually use it. I’ve been using it for about a year now on a sustained basis. It has simple emacs keybindings, like other OS X text editors, so that’s nice for me too.

For ideas best expressed by complicated freehand drawings, I use pen and paper. I always carry an unruled (no lines) notebook for this purpose.

For a while I was using a small drawing tablet and Curio (http://www.zengobi.com/products/curio/) for drawings, but it didn’t stick. The GUI was a little too slow, and plugging in the tablet was too much of an extra step. A tablet Mac would solve this. (Yeah, yeah I could get a PC, but I’d rather avoid it if I can.)

When I take notes at a meeting or a talk, I use TextEdit (again, w/ simple emacs keybindings), and depend on spotlight to help me relocate things. I prepend all filenames with the date in <2 digit year><2 digit month><2 digit day> format, so by default things sort by date across filesystems etc. This is surprisingly useful.

Luminotes.com or Google Notebook.

On the go? Moleskine

I was in the same boat, taking at least 10 notes in a session, several times a day. I work best by blasting notes in and sorting through them when I come up for air. There wasn’t anything that allowed me to work this way very effectively, so I did what any of you would have done – started a startup, of course!

I’m currently trying different systems, and to be honest, I’m not 100% happy with any of them.

Started with the lovely and simple Notepad textfiles, but after a bit they are too simple and too hard to maintain, thus not good.

Then I used Google Notebook for a long time, was easy to add stuff, but not so easy to find it later. Plus it’s still too simple for my liking, can’t categorize ideas too well.

I switched recently to Evernote, which seems to be an improvement over Google Notebook, but for some reason I still don’t feel comfortable with it. The fact that I can type offline and sync with different computers or read my ideas online is really welcome and handy, though.

Everytime I fall back to my paper notebook, which is also too simple and not search friendly, but I like handwriting and for some strange reason, ideas flow much better than when I write rather than type them. The real only grudge I have with it is that there is no backspace key and no “insert a new line in the middle of the text” either😦

balsamiq is great for ui mockup. Other than that I agree with habs, paper works. Although i have recently started using Microsoft One Note 2007. It makes for great brainstorming / idea throwing / white boarding.

If you’re like me, you’re working on many things at once.You’re continually developing, doing research, fixing bugs, helping people, etc.When it comes time to do your status report, how do you remember what all you did this week?When you see a bug that looks a lot like one you fixed two months ago, how do you remember what you did to fix it?The solution is simple, and yet I find many developers don’t do it.Take notes.You should take short, but informative, notes as you work, on all the things you work on that day.These notes don’t have to read like the next Great Novel.The purpose of the notes is to trigger your memory into remembering what you were doing when you took the note.

This is really all you need.There are some specialized note-taking tools that work well, however.When I’m in a Microsoft Windows environment, the Microsoft OneNote program is excellent.It has facilities for putting notes in folders, within projects, as you would expect; but it adds things like being able take screen shots, and then index the text in that screen shot so that you can search on it later.That’s useful for when an error message pops up in a dialog box and you need to capture its contents.You can highlight text, tag paragraphs, etc. etc.But for 95% of my professional life, I’ve just been grepping text files, and like I said, that’s really all you need.The important thing is that you capture the information.

I couldn’t agree more. I’ve been taking notes like this since a couple of years and they have been a real time-saver.

I have never used OneNote, but I’m eager to suggest WikidPad as a great alternative: open source, actively developed, no-nonsense, written in Python and customizable to the bone.

I find the Harvest Co-op app (http://coopapp.com) to be ideal for this. It is like twitter for teams, and if you use Harvest for time tracking Co-op allows you to track time, too, by linking to your harvest account. Handy.

I started seriously taking notes when I switched from networking to programming in 2005. It took a while to find a tool I liked but I came across Confluence and have been very happy with it since then.

I’ve been taking notes like this for about three years now and I have to admit I still haven’t got it perfected yet. In the beginning I took verbose notes detailing everything I did, but after a few months it became unwieldy to go back and manually search through a hundred pages of notes trying to remember where I might find that detail I knew I had seen.
It was a little better when I finally started doing everything on the computer, but invariably it seemed like I was recording a lot of information that I never ever needed again, and spending a lot of time doing so, and when I did need some tidbit of data it wound up being the one thing during the day in question that I had considered unimportant or for whatever reason had not documented.
I’ve sort of had to resign myself to simply keeping a whiteboard with items that really stand out, such as stupid bugs that I fight for half the day only to find there was some tangential solution that really should be unrelated to what I am doing (”oh! my program isn’t working because the router needs to be power cycled!”).

I’ve been using OneNote for a little while now for tracking daily status for stand-ups, and it’s pretty much what I need – good WYSIWYG editor, search, groups, etc.

Picking the right tool up-front seems pretty key, cause once you have a few months of notes written, it’s tough to switch.

Good post.

I’ve been trying to do this for a while. Getting into the habit is the hardest part. I tried a paper notebook a week or two, and then made a Google Docs Spreadsheet with a single entry form. It adds a timestamp to the spreadsheet automatically, and I’ve got it in my toolbar and on my phone home screen, so I can make entries anywhere.

I am the sole note-taker in the group which means that I get asked the questions about what we did and when. I use Evernote (non-synced) and record daily activities. I also set up a personal wiki on my server so that other people can access notes that I have taken. I have to use a windows machine at work so I use screwturn wiki (it was tough to get that unblocked!)

Also, time tracking with Rachota is a breeze! I have turned 2 of my 3 co-workers on to it.

For years I’ve managed to make this clutter work for me. But this spring, I noticed that too many of my to-dos were not getting done. (Take this assignment, for example. Between its official due date and the day I actually filed it, two and a half shameful weeks passed.) With a book to finish, a business to manage, and a big move to make, I decided that it was time to find and commit to a proper system of task management.