Windows and Linux both support PHP and mySQL. But Linux does not support ASP.net.
ASP is more of an application coding language than PHP which is web interface language. ASP allows you to do some more stuff, but most people wouldn't need any of these, for example when you update windows via web update from
www.update.microsoft.com you see the ASP.net working. You basically get the application window inside the web browser, PHP can't do that.
But there are also other very important differences. Windows has more security issues when not protected properly. Also Windows is slower and less stable than linux. Linux has greater support (Open source communities). Linux is free. Most of the linux's software is free, for example when you want to run a windows server you would have to buy windows itself, then you need to buy extra software, like msSQL to use databases etc.
And these are the reasons why 99% of web servers run Linux.
I would switch over to linux, mainly because linux servers are more secure and stable. Also if you're beginner then 99% of the tutorials concerning website creation are made for use with linux server.
EDIT: Oh, and for CMS I would suggest WordPress, I tried a lot of CMS's all the most popular ones as well as some new comers.. I find WordPress by far the easiest to work with. It has tons of plugins, it is easy to customize it, it is free..
And I would have to disagree about "You get what you pay for".. it may apply to "Real World" but @internetz in many cases free, open source software is by far superior to commercial competitor. With exceptions of course

Millions of dollars from companies help commercial stuff a lot.. which is the only reason for these exceptions.
I mean look at it..
Only reason I don't use GIMP is because it has very different interface from Photoshop and I simply don't wish to learn everything from the beginning again.
Only reason I don't use Linux as my main boot OS is that I would have to re-learn how to do everything.
How I wish that my first computer would have Linux instead of Windows.. when I would have as much experience with Linux as I have with Windows I would most likely be about 27x better coder, know 27x more about computers and have much faster workflow. And spend 27x less time on pointless stuff like.. figuring out why the hell Vista needs to ask every single thing twice or more from me..