The links probably have a target of
_blank at the moment, which forces them to open in a new window.
HTML Code:
<a href="http://www.amazon.com/" target="_blank">Buy something from Amazon in a new window</a>
If you remove the target attribute (or set it to
_self), they will open in the current window.
If you change the target attribute to
name (where name is any string that takes your fancy*), they will all open in the same, new window.
Edit: I should also have said that you can use as many different names as targets as you like. So if you have an Amazon shop and a Google shop (or whatever), you can use target="shopAmazon" to send all Amazon clicks to one window and target="shopGoogle" to send all Google clicks to a second one.
Alternatively, it's possible you have JavaScript opening the new window (via window.open). In that case, I'd probably suggest removing the window.open and replacing it with a standard link plus optional target attribute so those without JS (and, if it's badly configured, those who Ctrl-click) can use your site.
* Perhaps. You may be limited to [A-Za-z0-9]+ Try a single word if you have problems.