I loaded the pubkey with hubi's instructions OK, apt-get update OK.
However, never got to install anything after that, because I saw slam's warning.
I did as slam said because I did not notice any difference after having the Oracle repo installed, so I uninstalled it.
Both virtualbox and virtualbox-ose are in repo.
Now, if I install virtualbox-qt it will uninstall virtualbox 4.1.2-73507, which is latest version from Oracle for Debian, and install an older version of virtualbox, which is dfsg.
Basically, all other available packages are for dfsg version, only Oracle package is the one mentioned.
There is a mix-up in repo, that is why I thought of downloading the latest version from Oracle and installing with gdebi.
Your suggestions?

I don't know what's the difference. Two years ago or so it was all about handling USB. If you pull virtualbox from Debian, forget the packages with -ose, they are just metapackages as it's called virtualbox in Debian as well now. Debian packages should build with new kernels all the time, I am not sure about those from Oracle.

I switched from the Oracle packages to the Debian packages half a year ago, I cannot see any differences for what I am using the VM. But I don't use USB devices with that box inside the VM.

There is no difference besides the Version and the packaging.
So, if non-free features are needed (usb2, vbox-rdp), then in both versions is need to install the extpack.
I see no reason anymore for using the packages from virtualbox.org.

Actually there is no difference in packages offered, both the Oracle website and the repo offer virtualbox-4.1_4.1.2-73507~Debian~squeeze...according to architecture.
What I am saying is that if I follow slam's suggestion, packages will conflict and the above mentioned version will be removed.
@ towo: if I install guest-additions-iso from repo it will conflict.
This is what I did on another Linux distro, I simply downloaded from virtualbox.org and installed with Gdebi, never had an issue with pubkey either because was not even accessing their repo.
I installed all other needed packages same way, virtualbox has a link within itself to install guest additions.
When an update was released I was alerted and sent a link to download.
I installed right on top of previous version, did not even save appliances and everything worked fine.
I'm wondering what route to take...

Still confused.
I cannot add the guest additions from repo, without it uninstalling latest vbox version.
However, this will not happen if I download and install the extension pack from Oracle.
I can also install dkms from Oracle.
Bottom line, I cannot add extensions without creating a conflict, either I install extensions for one, and stick with the Debian repo only, or I take those packages from Oracle.
Finally, there are two versions of virtualbox in repo:
1.
The above mentioned virtualbox-4.1_4.1.2-73507~Debian~squeeze...according to architecture.
2.
virtualbox 4.1.0-dfsg-2
I don't think that:
'There is no difference besides the Version and the packaging...'
There must be some other difference because the extension pack detects 1. and attempts to uninstall it.
Somebody willing to shed some light on this issue?

@ hubi:
Yes, that is precisely what I was saying, when I used to download and install from virtualbox.org I never needed anything else, "virtualbox has a link within itself to install guest additions."
Regarding dkms, I can also download that from Oracle, but never needed it, dunno because I was using Linux Mint Main which is based on Ubuntu, not Debian.
Since then I moved to Linux Mint Debian, ditched Ubuntu, and from there to Aptosid and Debian itself.
Maybe I will have to add that in the case of a kernel upgrade.
Any other comment?