According to the book Eragon, Utgard Mountain is so steep that the peak can only be reached through flight. Somewhere on the face of the mountain is an old Dragon Riders' outpost, built following the Palancar Crisis. The outpost at the time of the Rider War was nothing but a ruin.
Utgard has a small role in the Eragon chapter "Therinsford" after Eragon and Brom got their horses from the town of Therinsford. Eragon and Brom have to round the base of Utgard and descend out of Palancar Valley. At this point in the book, Eragon has a small revelation, realizing that he has become part of the ancient order of Dragon Riders.
"An outpost of the Riders - one that has lasted since their founding. That was where Vrael took refuge, and where, through treachery, he was found and defeated by Galbatorix. When Vrael fell, the area was tainted. Edoc'sil, 'Unconquerable,' was the name of this bastion, for the mountain is so steep none may reach the top unless they can fly. After Vrael's death the commoners called it Utgard, but it has another name, Ristvak'baen - the 'Place of Sorrow.' It was known as such to the last Riders before they were killed by the king."