In 1946 he was elected governor of Kansas. As governor, he pushed mental health programs as well as a long-term highway project. In 1949, Kansas senator Clyde M. Reed died, and Carlson appointed Harry Darby to fill the seat. Darby continued his service in the Senate unitl Carlson himself was elected to fill the seat in 1950. Instead of waiting until January to be sworn in, he took his seat on November 28, 1950 (it is very common for someone who is elected to a Senate seat that is at the time occupied by an unelected appointee to be sworn in early), leaving the office of governor to Frank Hagaman who served less than two months.

In 1952, he campaigned to get fellow Kansan Dwight D. Eisenhower into the White House, and then brokered a deal through Ohio Senator Robert Taft, known as "Mr. Republican" for his leadership of the party's right-wing, became majority leader. According to Billy Graham's biography "Just As I Am," Carlson organized the first Presidential Prayer Breakfast, later known as the National Prayer Breakfast and sponsored behind-the-scenes by The Family (Christian political organization). In 1955, Carlson coined the phrase "worldwide spiritual offensive" as a description of Cold War evangelical aims. In 1950, Carlson traveled to Haiti on behalf of The Family and returned to urge Congress to support the regime of the dictator Papa Doc Duvalier. Carlson was re-elected twice, in 1956 and 1962, before returning to Concordia for retirement.

Carlson died in 1987 in Concordia, Kansas and was buried in Pleasant Hill Cemetery in Concordia. A State judicial building was named in his honor, and US 81 from Concordia to Salina is named the Frank Carlson Memorial Highway. The local library in Concordia, the "Frank Carlson Library", is named in his honor. Carlson is the only Governor of Kansas to have served in both the United States House of Representatives and the United States Senate.