Both are first rate, but the aren't competitors.
ActiveForums is a forum module, and competes with the DotNetNuke forums that we are using right here.
Ventrian PM is for sending private messages (that's the PM part) from one person to another.
And, it's easy to get them to play nice together. ActiveModules also has a messaging utility that goes with ActiveForums.
And, if you're interested in ActiveForums, there is a new version (and a new generation) -- Version 4.0 -- due to drop momentarily. It should be quite a step forward for DotNetNuke forum modules.