A Whig and a Know-Nothing before he joined the Republicans, he became more radical on slavery and reconstruction as he grew older. In 1858 he was suspiciously friendly to Mr. Lincoln's opponent in the Illinois Senate Race, Stephen Douglas. Colfax supported Edward Bates before the 1860 Republican National Convention and sought unsuccessfully to deliver Indiana's delegates to Bates. Colfax himself was supported by Chicago Tribune for Postmaster General in 1861, but Indiana's spot on Cabinet went instead to Caleb Smith—souring Colfax on Lincoln and contributing to Colfax's shift to Radical Republicanism. Colfax wrote in January 1861 that "Mr. Lincoln said last week that with the troubles before us I could not be spared from Congress—that a new & untried man would fill my place...that Smith had nothing, while I was in office."
When Colfax sought the House speakership, President Lincoln initially opposed him, according to historian William Ernest Smith: "...the President begged Montgomery Blair to help him defeat the radical Colfax for Speaker of the House of Representatives in December, 1863. He considered Colfax a little intriguer. Neither Blair nor Welles trusted Elihu Washburne, whom the President preferred to Colfax, but Blair ran the risk of losing the friendship of Colfax to do the bidding of the President. Colfax claimed a friendship of long standing with the Blairs, and upon discovering that the Blairs opposed him for Speaker, he was, in the words, of [Gideon Welles], 'exceedingly sore.'"
A hard-working and able debater, "Smiler" was best known for his affability but not his erudition. Tee-totaler Colfax held well-attended social events at his home from which the liquor-loving often departed in search of less abstemious homes. Although a talented writer and speaker, Colfax's education had ended at age 10. Journalist Noah Brooks noted that "it was a cause of mortification to some that when the President's private secretary appeared at the door of the House with a message, he was invariably addressed by the Speaker a. Sekkertary.'"
Colfax was present at the White House the night the Emancipation Proclamation was signed on January 1, 1863. After noting that his hand had been shaking a bit when he signed, Mr. Lincoln told Colfax it was not "because of any uncertainty or hesitation on my part; but it was just after the public reception, and three hours' hand-shaking is not calculated to improve a man's chirography. The South had fair warning, that if they did not return to their duty, I should strike at this pillar of their strength. The promise must now be kept, and I shall never recall one word."
As the House Speaker, Colfax also participated in the ratification debates for the 14th Amendment, saying it would become "the gem of the Constitution," because "it is the Declaration of Independence placed "immutable and forever" there.
Colfax recorded the President's response the day after the Battle of the Wilderness in 1864: "I saw him walk up and down the Executive Chamber, his long arms behind his back, his dark features contracted still more with gloom; and as he looked up, I thought his face the saddest one I had ever seen. He exclaimed: 'Why do we suffer reverses after reverses! Could we have avoided this terrible, bloody war! Was it not forced upon us! Is it ever to end!' But he quickly recovered and told me the sad aggregate of those days of bloodshed....An hour afterward he was telling story after story to congressional visitors at the White House, to hide his saddened heart from their keen and anxious scrutiny."
On other occasions, Mr. Lincoln did not hide his heart. One day, the President said to Colfax: Some of my generals complain that I impair discipline and subordination in the army by my pardons and respites, but it makes me rested, after a day's hard work if I can find some good excuse for saving a man's life, and I go to bed happy as I think how joyous the signing of my name will make him and his family and friends."
Colfax was close to Secretary of the Treasury Salmon Chase but was circumspect in any support he offered to Chase's ill-fated presidential bid in early 1864. The House speaker was allied with Radical Republicans but never made himself personally obnoxious to Mr. Lincoln as many others did. Colfax did not join the cabal of worried Republicans who wanted Mr. Lincoln to get out of the race in August 1864 but in September he urged reconsideration of the President's draft call, believing it would hurt Republicans in the October elections in Indiana.
After the 1864 election, Colfax lobbied hard for Chase's appointment to become chief justice of the Supreme Court. In early December, Colfax wrote Chase: "Friday night I had a long talk with the President. He told me of various objections he had heard, which he added, did not influence him, but he repeated them. Some were sinister ones, whose paternity I think I can recognize, & which, I told Mr. Lincoln were unworthy of their author. Another was that your ambition was the presidency, & that a Presl. Candidate should not be C.J. to use it as a steeping stone, impairing the strength and impartiality of the Judiciary. I asked Mr. Lincoln if you had once told him you preferred to be C.J. rather than president. He replied Yes; and I added that, if appointed, I felt certain you would dedicate the remainder of your life to the Bench."
Colfax visited President Lincoln on the day he was assassinated and Mr. Lincoln expressed his envy that Colfax was leaving on a trip to California. According to journalist Noah Brooks, he met Colfax after he left the White House and he "tarried with me on the sidewalk a little while, talking about the trip." President Lincoln had also told Brooks that he had considered moving to California because it would offer opportunity for his sons and "when he heard that Colfax was going to California, he was greatly interested in his trip, and said that he hoped that Colfax would bring him back a good report of what his keen and practised observation would note in the country which [Colfax] was about to see for the first time."
It was probably sometime between 1863 and 1873 that his name got put on a major Denver thoroughfare. He visited the city in 1865, 1868 and 1873. Little-known Denver pioneer Daniel Witter, who owned considerable real estate near the city, was the federal tax assessor. (Colorado was a territory then, so all the good political jobs resulted from federal patronage.)
Witter's wife, Clara, was Colfax's step-sister, and as one city history recounts, they wanted to make sure he enjoyed his trip to Colorado. "Anxious to please the goose who laid the golden egg of a federal job, the Witters paid $2.50 a dozen for eggs when Colfax visited in 1868 so that, as Clara explained to her children, 'brother Schuyler could have his boiled egg for breakfast."'
Alexander Cameron Hunt, was appointed as Territorial Governor on April 24, 1867. Hunt's administration was dominated by conflicts between Indian tribes, primarily the Utes and other Plains Indians, and white settlers. The Indian problem became so severe that Central City offered money for Indian scalps, a measure that probably did not help the situation. Tension grew with word that the native Ute Tribe was seriously considering joining efforts with other Plains Tribes, a prospect that made people living in settled areas fear for their existence. Hunt's previous relationship with the Utes as Ex-Officio of Indian Affairs, however, helped to relieve this tension. In February of 1868 the Ute Chiefs, Indian agents, and President Andrew Johnson were brought together by Hunt to discuss a possible peace treaty. An agreement was reached which forced the Utes to vacate all lands east of the 107th meridian. The federal government chose not to honor their end of the bargain by reneging on the promised food and products. Understandably the Utes were upset, and began to strengthen their ties with the Arapahoe and other Plains Indian tribes. Hunt personally defused the situation by traveling from camp to camp and discussing the problem with the Ute leadership.
An agreement with the Utes, however, did not dissuade the Arapahoes. The following August, Schuyer Colfax, Samuel Bowles, Governor Bross of Illinois, Governor Hunt, and several prominent Denver families went camping near Buena Vista. Meanwhile, a party of Arapahoe warriors was seen traveling toward the campground. Territorial Secretary Frank Hall quickly created a volunteer army which was sent out to protect the camping notables. Luckily a messenger reached the Colorado Governor first so that Hunt was able to utilize his friendship with the Ute Tribe and obtain an armed escort back to Denver. When Colfax returned to Washington he pushed officials to create a larger military presence in the Colorado Territory, and praised the heroism of both the Colorado pioneers as well as the Ute braves who had escorted the camping group back to Denver.
In 1872, Colfax wanted another term as vice-president, but the Republicans denied him the nomination even while giving Grant another term. That was because Colfax was implicated in the Credit Mobilier scandal.
Thus ended the political career of Schuyler Colfax. But his name lives on, even though no one seems to know just what political favor Colfax performed in exchange for getting his name on Colorado's most prominent street, nor why no one felt enough shame over the scandal to propose renaming it.