If you chose not to tinker around with your Windows operating system, then the file automatically goes in the Recycle Bin. If you ever decide you still need the file, you can simply search through the trash (a funny concept Microsoft makes us go through) and you'll find your deleted file intact and easily recoverable. However, if you disable the Recycle Bin, things will go slightly different.
Most of us consider deleting a file permanent. After all, you can not see it in the directory structure so it's definitely not there. Unfortunately (or well, fortunately, depending on how you look at the situation) when it comes to computers, seeing is not necessarily a premise for believing. A file does not get permanently destroyed and eradicated from your memory by simply deleting it from the directory structure. Instead, it is kept safe in a free memory area, stored in case you change your mind and need your file back. What gets deleted is the link pointing towards the file, so if you would compare a file to a book page, deleting the file does not tear the page out of the book, but rather deletes the page's index from the table of contents, represented by the file system here.
Still, your hard disk space is not infinite, so eventually the space your deleted files gets crammed into will fill up and no more "closet" space will be available. What the operating system does in such a case is overwrite the old deleted files with newer deleted files, so there's a certain time limit in which you could recover your lost data. For example, if you deleted a file from your hard drive a year ago, the chances are slim that you'll find it intact in the "closet" area. Instead, if you deleted it yesterday there's no chance it got overwritten.
Of course, data recovery techniques have been constantly improving and specialists can now recover the data even after it has been overwritten! Some companies boast the fact that they can recreate files that have been deleted and subsequently overwritten in the lost memory sector by up to 10 times. However, after 10 overwrites, there's a good chance that the data that is recovered is not entirely intact and some parts might be missing.
Just like there are programs that help you recover deleted data, there are other ones that help you permanently delete it! Sometimes you'll want to permanently destroy some data you stored on your PC and your closest resemblance to a paper shredder is such a data deletion tool. As data recovery technology is on a constant rise, there's a chance bits and pieces of your "permanently deleted" data can still be found, so it's best not to count entirely on that if you really, really do not want someone to find out what you stored on your hard disk.