Extreme orbital evolution from hierarchical secular coupling of two giant planets

Observations of exoplanets over the last two decades have revealed a new class of Jupiter-size planets with orbital periods of a few days, the so-called "hot Jupiters". Recent measurements using the Rossiter-McLaughlin effect have shown that many (~ 50%) of these planets are misaligned; furthermore, some (~ 15%) are even retrograde with respect to the stellar spin axis. Motivated by these observations, we explore the possibility of forming retrograde orbits in hierarchical triple configurations consisting of a star-planet inner pair with another giant planet, or brown dwarf, in a much wider orbit. Recently Naoz et al. (2011) showed that in such a system, the inner planet's orbit can flip back and forth from prograde to retrograde, and can also reach extremely high eccentricities. Here we map a significant part of the parameter space of dynamical outcomes for these systems. We derive strong constraints on the orbital configurations for the outer perturber that could lead to the formation of hot Jupiters with misaligned or retrograde orbits. We focus only on the secular evolution, neglecting other dynamical effects such as mean-motion resonances, as well as all dissipative forces. For example, with an inner Jupiter-like planet initially on a nearly circular orbit at 5 AU, we show that a misaligned hot Jupiter is likely to be formed in the presence of a more massive planetary companion (> 2 MJ) within 140 AU of the inner system, with mutual inclination 50 degrees and eccentricity above 0.25. This is in striking contrast to the test-particle approximation, where an almost perpendicular configuration can still cause large eccentricity excitations, but flips of an inner Jupiter-like planet are much less likely to occur. The constraints we derive can be used to guide future observations, and, in particular, searches for more distant companions in systems containing a hot Jupiter.