As a general rule, we like to have automated tests for everything that
can be practically tested. However, there are certain types of bugs
that are best caught with old fashioned manual testing (also called
manual QA). Manual testing not only catches bugs, but it also helps
developers learn more about the system and think about the existing
semantics of a feature they’re working on.

This doc assumes you know how to set up a local development server
and open the Zulip app in the browser. It also assumes a basic
knowledge of how to use Zulip.

Always test with multiple users (you can use incognito windows
to facilitate this).

Always keep the inspector console open and watch for warnings
or errors.

Be methodical about collecting information on bugs. (You will
eventually want to create tickets, but you may want to consolidate
your own notes before filing tickets.)

You generally want to test with Cordelia as the primary user,
and use Hamlet as her primary conversation partner. Use Iago
when you need to test administrative functions. Send messages
to Othello or Prospero if you want to verify things such as
Cordelia not being able to receive messages not intended for her.

The rest of this document groups tasks into basic areas of
functionality of the system. If you have multiple people testing
at once, you can divvy up QA tasks by these sections in the doc.

Zulip uses the term “narrowing” to refer to opening different views
of your messages, whether by clicking on sidebar options, recipient
bars, or by using search. The main focus of these tasks should
be watching unread counts. Of course, you also want to see messages
show up in the message pane. And, finally, you should make sure
that no messages outside the narrow show up in Cordelia’s view.

Important: Make sure that Cordelia is subscribed to Verona but not
subscribed to Denmark; if not, you should use different streams
for your testing.

When testing narrows, you want to have Hamlet send the same message
several times in a row, while cycling Cordelia through various narrows.

Here are the main tasks for Hamlet (and each message gets sent several
times):

Send Cordelia/Othello a PM.

Send Cordelia a PM.

Send Othello a PM.

Post to Verona/foo.

Post to Verona/bar.

Post to Denmark/foo.

Post to Denmark/foo and mention Cordelia.

For each of the above types of messages, you will want to cycle
through the following views for Cordelia (and have Hamlet send new
messages after each narrow):

Go to All Messages view.

Go to Private Messages view.

Go to Private Messages w/Hamlet.

Go to Private Messages w/Hamlet and Othello.

Go to Verona view.

Go to Verona/bar view.

Go to Verona/foo view.

Go to Denmark view.

Go to Denmark/foo view.

There are 56 things to test here. If you can get into a rhythm
where you can test each case in about 30 seconds, then the whole
exercise is about 30 minutes, assuming no bugs.

We have pretty good automated tests for our markdown processor, so
manual testing is targeted more to other interactions. For composing
a message, pay attention to details like what is automatically
populated and where the focus is placed.

Hotkeys

use r to reply to a stream message

use r to reply to a PM

use R to reply to the author of a PM

use R to reply to the author of a PM stream

use c to compose a stream message

use x to compose a new PM

Buttons

Narrow to a stream and click on “New topic”

Narrow “Private Messages” and click on “New topic”

Narrow to a stream and click on “New private message”

Narrow “Private Messages” and click on “New private message”

Topics

Compose/send a message to a stream with no topic.

Compose/send a message to a stream with a new topic.

Compose/send a message to a stream with autocomplete.

Compose/send a message to a stream manually typing an
existing topic.

Formatting stuff

Use the “A” icon to get markdown help.

Use the eyeball icon to show a preview and send from preview mode.

Toggle in and out of preview before sending a message.

Use @-mention to mention Hamlet (and send him a message).

Use #**devel** syntax and send to Hamlet, then follow the link.

Create a bulleted list.

Use the emoji icon to find an emoji in the picker.

Attachments

Send a message with an attachment using the paperclip icon.

Send a message with multiple attachments.

Copy an image from the clipboard.

Use drag/drop from the desktop to upload an image.

Drafts

Start composing a message then click outside the compose box.

Use “restore drafts” to restore the draft.

Start composing then use “Esc” to abort the message.

Use “restore drafts” to restore the draft.

Start composing a stream message and then abort using
the little “x” icon in the compose box.

Click on “New private message” and restore the draft. (You
should now be sending to a stream.)

For this task you just want to go through all of our popover menus
and exercise them. The main nuance here is that you occasionally want
to click somewhere on the UI outside of an existing popover to see if
the popover menu is “too sticky.” Also, occasionally actions will be
somewhat jarring; for example, if you mute a message in the current view,
then the message will disappear from the view.

Here are the things to test:

Stream sidebar menus

Stream settings (just make sure it goes there)

Narrow (and then have Hamlet send a message)

Pin/unpin (do both)

Compose (send a message to the stream)

Mark as read (scroll back and then have Hamlet send you a message)

Mute/unmute (do both)

Unsubscribe (and then go to Stream settings in the gear menu to resubscribe)

Choose custom color (play around with this)

Topic sidebar menus

Narrow (and then have Hamlet send a message)

Mute/unmute (try both)

Mark as read (scroll back and then have Hamlet send you a message)

Left-message-pane menus (click on person’s name)

Verify email

Verify date message sent

Send a PM (make sure compose box is filled out ok)

Narrow to PMs with

Narrow to PMs sent by

Right-pane-pane menus (click on chevron when hovering)

use “i” hotkey to open the menu

Edit a message you sent (using the down-arrow key to navigate the popup)

The main task for testing search is to play around with search
suggestions (autocomplete). Once you select an option, verify the
message view is consistent with the search and that the left sidebar
reflects the current narrow. If a search comes up legitimately
empty, have Hamlet send a message that matches the search.