You can create whatever Terms you can think of a use for, grouped into whatever Vocabularies you like. There are infinite possibilities – which always makes it hard to get started.
It's difficult to immediately envision all the ways to set up your taxonomy, and all the possibilities for using it. Here's a general suggestion for starting out:
Sit down and do a little thinking about what keywords you may want to call upon in the future. You'll probably want at least two main groups of keywords (and thus, at least two Vocabularies): one that describes what a piece of content is, and one that describes what the content is about. (At least, I've always found that useful.) See examples below. You may want additional Vocabularies, depending on what your site is about.
Next, fill each Vocabulary with Terms. If your site already has some Terms set up, delete any you know you won't need, and add Terms you think you will need. If in doubt, add a Term; it's easier to add it now and ignore (or even delete) it later if it turns out unnecessary, than to add it later (and belatedly tag all the old content that you now want that Term attached to).
Finally, on the technical side: If you're not worried about other users tagging content in odd ways, go for flexibility. Allow your Vocabulary to use the "Multiple select" option, and maybe skip the "Required" option. Consider free tagging too.
In summary: You can change your taxonomy and its workings any time down the road, but if you have a lot of content already in place, you'll have a big job going back and re-tagging all those old nodes. A little strategizing up-front can save you a lot of work later.
Example taxonomy: generic site
Here's a simple set-up that works for a lot of sites:
Vocabulary 1: Content Type
This Vocabulary is for Terms that describe what the content is. Not node type (Page, Story, Blog Entry, etc.), but its intended purpose within your site. Fill this Vocabulary with Terms like 'news', 'general info', 'essay', 'report', 'article', and so on. Some child Terms may make sense: 'news', for example, could have child Terms 'announcement' and 'press release' under it.
Vocabulary 2: Content Topic
This Vocabulary is for Terms that describe what the content is about. More than the above Vocabulary, appropriate Terms here will vary widely by site. Fill it with Terms that make sense for yours: 'product', 'company', 'website', 'celebrities', 'hobbies', etc.
For more detail, add child Terms. For example, the broad Term 'product' could have product categories (or specific products) under it as child Terms.
Using the Terms
The above two Vocabularies should enable pretty powerful organization of content. Any content should fall under a combination of one Term from each vocabulary (or more than one from each).
For example, a notice about an upgrade to your company's website would use the Terms 'announcement', 'company', and 'website'. How about general company background? The 'General info' and 'company' Terms. A case study on a new product? 'Report' and 'product'.
Example taxonomy: recipe site
As another example, a site for collecting recipes might do well with this taxonomy:
Vocabulary 1: Content Type
Same as above, but recipes themselves are key enough to this site to be considered a major type of content, not just a topic under 'general info' or some such. Inside the Vocabulary called "Content Type", add a new Term, 'recipe'. (You'll also want to decide on a consistent node type to use for recipes, such as Page, Story, or even a newly-created Recipe node type.)
Vocabulary 2: Recipe Type
Fill this with Terms like 'main dish', 'desert', 'appetizer', etc.
Vocabulary 3: Recipe Style
Fill this with Terms like 'Chinese', 'French', 'Mexican', etc.
Using the Terms
With this set-up, it's easy to see how your site can quickly call up, say, all Chinese main dish recipes (via the Terms 'recipe' + 'main dish' + 'Chinese'), or any other combination. Bon appetit.
Setting up your Vocabularies and Terms can take a little effort, as does remembering to tag content as you create it. But as your content grows, the ability to locate and organize it via those tags will make the work well worth it.