Intellectual and “intellectual” are two different things. If you actually mean “intellectual”, which is to say, not actually intellectual but making pretense at it, or possibly accused of it, that would probably be either Vampire: the Masquerade (with it’s many misused sesquipedalian words and overblown claims of uniqueness) or Immortal: Invisible War (which really wants to be Deep and Sophisticated).
With that out of the way, I’m going to assume the real question is about games that actually are intellectual, without the scare quotes. It really depends how you measure this. Aria: Canticle of the Monomyth had some serious research baked into the game rules & structures, though most people miss this because they’re so busy complaining about the nonstandard—and sometimes overblown—terminology.
Sign in Stranger and Shock: are both amazing, thought-provoking translations of real science fiction to the RPG medium—something that is rare among so-called scifi literature, even. It’s hard to play a game of either without stretching your point of view a bit.
Continuum is the RPG that has best tried to take the ramifications of time travel seriously, and not in a wibbly-wobbley, timey-wimey sort of way. As such, the mere act of playing looks to be mental exercise, keeping track of all the out-of-order events and how the characters relate and in what order those interactions occur.
HiT, which only ever made it to an “alpha limited” edition, as far as I know, is a different kind of intellectual. It attempted to build a fractal game system, with varying levels of detail & complexity, around the Threefold model of RPGs. It largely succeeded, and surprisingly made for a very approachable and playable game—unlike some discussions of game theories.
However, I think that the game that best fits my notion of what intellectual means in the context of RPGs would be Nobilis, the game where you can play the sentient embodiment of an idea, ideal, or concept. So your character might be “the embodiment of rain” (easy to imagine appearance and powers—but what’s your motivation?) or “the embodiment of the ideal of trust” (now personality and maybe motivation are easy, but what does that look like?) or “the manifestation of acrophobia” (good luck!).