New agile board not porting existing project tasks

Hello, I'm starting a new sprint and have added all my tasks to a project. I am now going to the Agile menu to start a board, the tutorial suggests using a filter to see tasks for an existing project. However there are not instructions on how to do that. If I go into a task and edit it to say "in progress" I can see the task in the Agile board. However it defaults to the current sprint, not the backlog. If I have it set for "not started', it will not show on the backlog or in the current sprint at all.

Originally I had set up a sprint with the tasks, logged out came back and the even though the board was there, I couldn't view the tasks. I thought the filter was the problem, deleted it and wound up deleting the sprint board. I really don't want to recreate the wheel.

I looked in the help and there wasn't an answer, and couldn't find an answer here as well.