I always bought scrapbooking stuff at AC Moore, you might look at their online store.
To start I would get a book, some different kinds of pages, packs of stickers, photo corners (little sticky things that go on the corners and keep them in the scrapbook or the little tabs that go on the back of photos, scissors that cut in different designs, some different colored paper, glue stick... probably some other things I am not thinking of.
Obviously you dont need everything listed but those are some ideas!