Unless you are modifying the core, there's no reason you can't do both. I would recommend running DNN 4 as your code base. Then, if you want to write custom modules, you can do that with VS2003 and compile against the DNN3 dll. There are some differences in how you do things in VS2003 and VS2005, but most DNN3 modules run fine in DNN4. If you can upgrade your development environment at a later date and not have to worry about upgrading your core, you will save yourself A LOT of stress.
Lessons learned from personal experience...
If you use 3rd party components, like Telerik RAD Controls, you should be aware they have a .NET1 and .NET2 version. If you use VS2003 to build modules and use a control (say RAD Grid) with the .NET1 version and try it install it in DNN4, it will work fine so long as you have the correct radgrid.dll in your bin directory. The problem comes in when you have some modules using .NET1 and others using .NET2 versions of the same control. When you do that, you will get an error stating something like radgrid.dll is ambiguous, meaning it's confused by two versions of the same control living in your bin directory. My advice is either use generic controls, like the regular datagrid, or be prepared to upgrade those modules when you make the switch to VS2005. In most cases, upgrading the module is pretty easy. For me, it was as simple as replacing the references and tweaking a few things. Your experience may vary but as long as you know ahead of time, you'll be prepared when the time comes.