For more on this teaching, see Yalkut Shimoni, “Korach”, Remez 752, beginning with the word “va’yakhilu”.
The full quote of this passage in Bamidbar Rabbah (18; 4), reads as follows:
“Rabbi Nathan said, ‘The Holy One, Blessed is He, said, ‘If all of the sorcerers of the world would gather and seek to change the morning into the evening, they would be unable [to do so. So too], just as I have separated between light and darkness, I have similarly separated Aharon to sanctify him in The Holy of Holies.’ Immediately, “And Moshe sent to call to Dathan and Aviram the sons of Eliav, and they said, ‘we will not go up.’” (Bamidbar: 16; 12) The [pasuk] here does not [state], ‘we shall not go’ and ‘we shall not come’, rather, [it states], ‘we shall not go up’. [The verse in Mishlei states], “The mouth of a fool brings ruin to him, [and his lips are a snare to his soul.]” (Mishlei: 18; 7) - [Dathan and Aviram] opened their mouths to punishments, to say that they will die by descending and not by ascending. [In accordance with that which] they [responded to Moshe], so did they die, “and they and all that belonged to them descended alive to sh'ole...”” (Bamidbar: 16; 33)