“The enemies of Torah scholars” in our context refers to the “Torah scholars”. When the gemara states something negative about the Jewish People or Torah scholars, for example, it will write in “lashon sagi nahor” – “language of a multitude of light”. This phrase is often used when it refers to something negative happening to the Jewish People or Torah scholars. Such as “if not for… the “enemies of Israel would be liable to be destroyed”. As the gemara does not want to say such a negative thing directly in reference to Israel, and rather euphemistically refers to “Israel” as “the enemies of Israel”. In our context, the gemara states “a sword upon the enemies of the Torah scholars”, euphemistically referring to Torah scholars.
 The full passage in the gemara (Berachos, 63b), is found below:
The passage discusses the pasuk (Divarim: 27; 9), “…pay attention and listen Israel, today you have become a nation to HaShem, your G-d.”
On the word “הסכּת” – “haskeis” – “pay attention”, the gemara states, ““Haskeis”: Make groups and involve yourselves in Torah [study], for the Torah is only acquired in groups [who study Torah]. This follows [that which] Rabbi Yosi [said] in the name of Rabbi Chaninah. For Rabbi Yosi said in the name of Rabbi Chaninah, what is the meaning of that which is written [in sefer Yirmiyahu], “A sword to the badim and they shall be made foolish…” (Yirmiyahu: 50; 36) – A sword upon the enemies of the Torah scholars who each sit alone (bad b’vad) and involve themselves in Torah study. Not only [is this the case], but they [also] grow dull. It is written over here [in the pasuk quoted from Yirmiyahu], “v’noalu” – “and they shall be made foolish”, and it is written over here [in Bamidbar (12; 11)], “asher noalnu” – “that we have been foolish” [which Onkelos translates as “we have grown dull”]. Not only [do these Torah scholars grow dull], but they [also] sin, as it says, [immediately afterwards in the same pasuk], “and that we have sinned”. If you want, I could say that [the source for these Torah scholars coming to sin can be found] here [in sefer Yishayahu (19; 13)], “The ministers of Tzoan have grown dull…”, [thus causing Egypt to stray and suffer as a result].”
Note: RaSh”I notes on the word “הסכּת” – “haskeis”, that “has” means “make”. The Hebrew word “כּת” means “group”. Therefore, according to RaSh”I’s rendering, in the context of our gemara, “הסכּת” means “make groups [of Torah learning]”.