Go into start and in the search box type "cmd". Press enter when the items appear from the search.
You will see a black window pop up. Type in "ipconfig" without the quotation marks and look for "default gateway". The number there is your router's IP address. Write that down somewhere.
For the following, make sure you are using the computer that is directly connected to the router via a cable or you may have problems (don't continue, post back and ill try to suggest an alternative). Open up your browser and type in the IP address you wrote down.
You will arrive at a page requiring an ID and password (you should know these of your network, if not, look for somepaperwork received with the router - the details will be there). Use these to log in and you can customize your settings from here.