head to head
Neither is simply harder - they are hard in different places. Pokemon X demands more reads; The Elder Scrolls V: Skyrim demands more execution.

Optimization · Pure Optimization
The first fully 3D mainline Pokemon adventure, introducing Mega Evolution and the Fairy type in Kalos.

Optimization · Pure Optimization
An open-world RPG mostly about discovering and stacking exploitable progression systems.
Pokemon X leans Optimization (Pure Optimization); The Elder Scrolls V: Skyrim leans Optimization (Pure Optimization).
The Elder Scrolls V: Skyrim leans clearly harder on execution.
Pokemon X
Menu-driven combat throughout
The Elder Scrolls V: Skyrim
Combat is forgiving; aim and timing rarely punish you.
Pokemon X leans clearly harder on reads.
Pokemon X
Scripted opponents; online battling optional
The Elder Scrolls V: Skyrim
Enemy AI is scripted and exploitable; not adaptive.
Pokemon X leans a little harder on variance, though the gap is small.
Pokemon X
Wild encounter and catch randomness
The Elder Scrolls V: Skyrim
Loot tables and leveled lists inject mild randomness.
Pokemon X leans a little harder on optimization, though the gap is small.
Pokemon X
Mega Evolution timing and team planning dominate
The Elder Scrolls V: Skyrim
Skill stacking, enchant loops, and exploits are deeply solvable.
A matching score does not mean a matching test. Two games can both sit at 10 on execution and demand completely different things of you - which is why the reasoning above matters more than the number beside it.