I guess the good news is that you don't really have to do anything as WordPress has done it all for you.
You should be able to see an RSS icon in your browser's address bar. Click it to view and subscribe to your feed. If you don't have an icon there for some reason, add a line like this to your theme's header
HTML Code:
<link rel="alternate" type="application/rss+xml" title="<?php bloginfo("name"); ?> RSS Feed" href="<?php bloginfo("rss2_url"); ?>" />
The default WordPress feed is available at yoursite.com/feed/ (if you have permalinks on). This is RSS 2.0 I think;
WordPress also provides Atom, RSS 1 and RDF feeds.
If you want to add text or an image somewhere to encourage people to subscribe, just link to that URL.
As you're going with FeedBurner, you should also get hold of the
FeedSmith plugin. This redirects your default RSS URL to FeedBurner and saves you having to edit the theme yourself.
And for reading RSS, the most popular program out there is
Google Reader. I personally use
FeedDemon, partly for historical reasons (Google Reader didn't exist when I first tried it

) and also because I like the way it works. Though you (and I) may be waaaay behind the curve -
here's what Robert Scoble had to say on the subject recently.