Gandalf appeared at first as one of the Maiar, as Olórin. He was the wisest of the Maiar, and learned pity and patience from Nienna – one of the Valar.
This meant he had the same order as the Valar, but to a lesser degree.
As a Maia, he would be a servant and helper of the Valar. The Elves don’t know how many of them there were, but they were seldom visible in form to Elves and Men.
Gandalf would later be known as Mithrandir, the ‘Grey Pilgrim’ among the Council of the Wise.
Council of the Wise
When the first shadows were felt in Mirkwood, there were the Istari – whom Men called the Wizards.
Mithrandir was the most vigilant and he doubted the darkness in Mirkwood. Out of Radagast, Curunír and himself – Mithrandir was closest in counsel with Elrond and the Elves. (Cúrunir was Saruman the White.) – this would later be known as the White Council, or Council of the Wise.
Gandalf had feared the first shadow of Sauron returning, and went to Dol Guldur. There was finally some peace for a while, however the Shadow returned and power increased, for a second time.
By then, the Council of the Wise had occurred consisting of Mithrandir and Curunír, along with Galadriel, Círdan and Elrond and other lords of the Eldar.
Additionally, Saruman had started learning about the Rings of Power, and the history.
As the shadow grew, the hearts of Elrond and Mithrandir had darkened. They realize that Sauron himself had taken shape again and was growing and trying to gather all the Rings to his hand.
Gandalf figured that with more power he could master the Enemy.
The Wise were troubled – but none were as perceived by Saruman the White – who went against the swift deeds to master Sauron suggested by Gandalf.
By then, Curunír had turned to dark thoughts and became a traitor. He did not want anyone finding the Great Ring, and started envying Sauron and wanting the Ring.
Mithrandir spoke to the council after a while – saying:
“It is not needed that the Ring should be found, for while it abides on earth and is not unmade, still the power that it holds will live, and Sauron will grow and have hope. The might of the Elves and the Elf-friends is less now than of old. Soon he will be too strong for you, even without the Great Ring; for he rules the Nine, and of the Seven he has recovered three. We must strike.”
JRR Tolkien, The Silmarillion “Of the Rings of Power”
Curunír assented – desiring Sauron to be thrust from Dol Guldur. He aided the council for the last time and Dol Guldur was assailed. But it was too late, and Sauron had seen it.
Quest of Erebor
“I was very troubled at that time,” said Gandalf, “for Saruman was hindering all my plans.” As Sauron arose and declared himself, and Gandalf didn’t know what to do, nor did he expect him there. (UT)
Gandalf had some dark thoughts, and he was away from the Shire for over 20 years. By then, Sauron had arisen again, and was close to declaring himself. Gandalf saw fit a need to attack Lórien and Rivendell at first. He urged the Council to see that Rivendell was not out of Sauron’s reach by then.
Gandalf came to believe that Bilbo was ideal for a task that he had at hand – before then, he met Thorin by chance in Bree, and they went to Thorin’s Halls. There, they discovered a shared desire to remove Smaug. Thorin had actually asked for Gandalf’s advice, so they went to the Blue Mountains. Thorin was mainly focused on “brooding of his wrongs, and the loss of the treasure of his forefathers, and burdened too with the duty of revenge upon Smaug that he had inherited,” for “Dwarves take such duties very seriously.” (UT, “The Quest of Erebor”)
And that was also when Gandalf had the idea of taking Thorin and his company of dwarves, along with Bilbo (and especially him) to travel to the Lonely Mountain.
Gandalf had been attracted by Bilbo during his childhood – as a young hobbit. “[Bilbo] had stayed in [Gandalf’s] mind ever since, with his eagerness and his bright eyes, and his love of tales, and his questions about the wide world outside the Shire,” …(UT, “The Quest of Erebor”)
…having heard news of him and saw that he was talking to Dwarves. Gandalf had tried to visit Bilbo many times, but he was often off, probably visiting the Dwarves. However, he went back to Thorin in order to persuade him to let Bilbo come along with him. “For Bilbo had changed, of course. At least, he was getting rather greedy and fat, and his old desires had dwindles down to a sort of private dream.” (UT, “The Quest of Erebor”)
Meanwhile, it was a map and key that would save the situation and get into Erebor, which was used to help Thrór and Thráin escape, according to Thorin.
Thorin was suspicious at the time, deeming Bilbo “soft as the mud of his Shire, and silly.” (UT: “The Quest of Erebor”)
Gandalf insists that “if [Bilbo] goes with you, you will succeed. If not, you will fail.” After many frustrations and concerns from Thorin, Gandalf eventually went through with his own foresight, and later decided to go to the Shire to visit Bilbo in TA 2941.
He would later follow Bilbo on his quest.
Gandalf during + after the War of the Ring
Gandalf showed up when Bilbo was celebrating his 111th birthday, and advised him to give up the ring and to give it to Frodo – which Bilbo eventually did. By then, Bilbo had already had the ring for 60 years.
Gandalf joined the fellowship as Gandalf the Grey in order to manage the beings of Middle-earth, and later would become Gandalf the White to research the technology of the enemy and become the leader.
He had borne the Ring, Narya. Because of this, he has seen further and deeper than any other in Middle-earth. However, it is not known that Gandalf would bear the Ring of Fire until he had arrived at the Grey Havens.