Following on Twitter can mean one of two things - Twitter users whose updates you subscribe to (people you are following) and Twitter users who subscribe to the updates you publish (people who follow you, called your followers).
Every time you write a new update (or "tweet") and publish it to your Twitter profile, it's available for the world to see (unless you set your account to make your tweets private). Inevitably, there will be some people who are very interested in what you have to say and will want to know whenever you publish a new tweet. Those people can select the "Follow" button on your profile page to subscribe to automatically receive your tweets. That means when they log into their Twitter accounts, their Home pages will be populated with a chronological list of the tweets of everyone that they're following, including yours.
The same holds true for people that you choose to follow. When you log into your Twitter account, your home page will show a chronological list of tweets from everyone you have chosen to follow by clicking on the "Follow" button on their Twitter profile pages.
You can choose to follow (or unfollow) any Twitter user you want at anytime. Since permission or approval is not required, Twitter is a highly open communications and networking tool.