Many people's first recording forays start with capturing the sound of their garden birds, and, depending on the location of your computer relative to your garden, if funds are tight you can save on the cost of the recorder by using your computer sound card as the recorder. Your computer should be newer than 1997, both from a general performance point of view and because standards for sound card inputs were established around that time.
You need to bear in mind that using a standard sound card isn't the best way to record birds. The mic inputs tend to be of mediocre performance. It is a matter of luck whether your sound card has a low enough noise level to record birds in your environment, but where finances are limited it is worth a go!
On the plus side, you have the advantage of fieldcraft and observation. You probably know your garden birds well - what fencepost the robin likes to sing from, or the blackbird. With a bit of ingenuity you can move your microphone closer to the birds or attract the birds closer to your mic by feeding, which does wonders for your signal quality. Suburban gardens aren't the quietest of environments, so you won't be trying to wrest the ultimate low-noise performance from your mic, so you may achieve a reasonable overall recording with a PC.
Listen to a couple of examples that were recorded using a PC, in this case with an outboard microphone amplifier.
Blackbirds start early at 4am. None were in my garden, most were several gardens away. I have amplified and filtered this recording, but it shows fixed recording has a reasonable range.
An hour later, the sparrows come along, but in the background you can still hear the blackbirds hold sway. A blue tit manages to quell the sparrows for a heartbeat. This is a straight copy of the recording - these are my garden sparrows and blue tit.
Start with what you have got - most PCs come with some sort of rudimentary microphone, which you can use to prove the recording side of things. Once you have that going, for best results you use the same kind of microphones and wind protection as you would going field recording, but plug this into your sound card, usually via a long cable. You can usually get 3.5mm stereo extension cables in store you would get other AV or PC components. The microphone jack of a PC is a slightly non-standard design but will work with self-powered mono mics with a mono 3.5mm jack such as the MKE300. Do not do this with a pre-1997 sound card, as the input jack was not particularly well standardised before then.
Although your are recording from a fixed location, in Spring you can often get some good recordings if you can set your PC to record early in the morning. The recording time is limited only by the PC hard disk capacity (or the operating system if you are using the FAT file system). You can set the recording going the night before and trim out everything before dawn. Birds start singing early, in my case from 4am in May and man-made noise does not follow the seasons. Traffic noise usually picks up from 6am, though the noise from short-haul aircraft movements sometimes starts earlier. You will be following an illustrious tradition - quite a few sound recordists early days started with a fixed mains powered recorder recording birdsong from a window!
You can improve the performance of the system by using a separate microphone preamplifier and feeding the output into the line input of your sound card. However, unlike in the past, portable recorders are now cheap enough that it is not really worth the cost to improve things in this way unless you have such gear already, or can build or borrow it.