The actual signing in in DNN Uses this method to authenticate (in VB for DNN)

System.Web.Security.formsAuthentication.SetAuthCookie(m.LoginName, false);

In order for the sub domain to read the same cookie as the top domain you have to se the cookie.Domain property to ".domain.com" this will tell all the sub domains to use the same cookie information ( I think the cookies called ASPNETSESSION or something.) The dot tells all sub domains to use this cookie.

Adding some code to the /admin/security/signin.ascx control at the end of the         
      Private Sub cmdLogin_Click(ByVal sender As Object, ByVal e As EventArgs) Handles cmdLogin.Click
method

like

//set the cookie to be used across domains..
HttpContext ctx = HttpContext.Current;
if (ctx != null)
{
   //set cookies to the top domain configured in the web config.
   foreach (string ck in ctx.Response.Cookies.Keys)
   {
      ((HttpCookie)ctx.Response.Cookies[ck]).Domain = ".domain.com"   //Config.GetSetting("Domain");
   }
}

Also your custom App will have to use the same authentication methods.. I think the cookie just stores the username.

You may be able to read the username from the cookie and log them in that way, or just use the same cookie to authenticate, it is just the standard windows authentication (I think), so using the windows methods may be the way to go.

courtesy of http://www.carlosag.net/Tools/CodeTranslator/Default.aspx

If you can't translate that, you may have some trouble implementing it, but good luck.