Does the present analysis of sentences to "clauses" (subordinate, etc.) has any roots/relatives in the classic grammar books (in Ars grammatica books, etc.)? I would be tankful for any hints or references.
Certainly the concept of subordination existed in antiquity; the Greek grammarians used ὑποβάλλω or ὑποτάττω to mean "to subordinate syntactically", and derived from this the term ὑποτακτικόν "subjunctive" (because the subjunctive mostly occurs in subordinate clauses). The Latin terms subordinare, subiungere, subiunctivum are calques of these. They also had at least two terms for specific types of clauses, namely protasis and apodosis, for the "if" and "then" parts of a conditional.
However, they may not have had a general concept corresponding strictly to our "clause". At any rate Apollonius Dyscolus (1CE), the most influential ancient writer on syntax, doesn't seem to have thought in terms of clauses: "Apollonius used no term that systematically corresponds to clause" (Allan 2006); he uses λόγος to refer both to complete sentences and to clauses within them. I don't know whether later writers on syntax (e.g. Dionysius Thrax in Greek, Varro and Priscian in Latin) improved on this terminology or not.
Hugh's informative answer appeared as I was typing this, and the terms he mentions can certainly describe what we think of as clauses. But as far as I know, none of them is a one-on-one equivalent of our concept of a clause: κῶλον is probably the closest in Greek (and membrum in Latin, which I assume is Cicero's translation of the Greek term), but a κῶλον sometimes describes a unit larger than a single clause, while a κόμμα can describe a short clause but can also describe a phrase that's smaller than a clause.
articulus ... Hence, a short clause, Dig. 36, 1, 27; also, a single word, ib. 35, 1, 4: articulus Est praesentis temporis demonstrationem continet, ib. 34, 2, 35: [Lewis and Short(TUFTS)]
membrum II B 3 Of speech, a member or clause of a sentence: quae Graeci κόμματα et κῶλα nominant, nos recte incisa et membra dicimus, Cic. Or. 62, 211; cf. Auct. Her. 4, 19, 26.— Lewis&Short
incisio a division, member, clause of a sentence, Gr. κόμμα: de eorum (circuituum) particulis et tamquam incisionibus disserendum est, Cic. Or. 61, 206: in incisionibus et in membris, id. ib. 64, 216; cf. incisum under 2. incido fin. C. L&S
In Short Clauses incisim, incaese, caesim
incīsim, adv. incisus, from 2. incīdo, I in short clauses (very rare):
Isidore of Seville. Book II xviii De colo, commate, et periodis. ... A phrase (comma) is a small component of thought, a clause (colon) is a member, and a sentence (periodos) is a rounding off.
(Translation and notes Barney, Lewis, Beach Berghof CUP 2006) this is followed by a clear example of phrase, clause and sentence close to the modern sense. Although Isidore is late , fl. 610 - 630, he is very conservative.
A comma is the marking off of a speech-juncture, as: “Although I fear, judges,...” –there is one comma, and another comma follows– “. . . that it may be unseemly to speak for the bravest of men, . . .” and this makes up a clause (colon), that is, a member, that makes the sense intelligible. But still the utterance is left hanging, and in this way finally from several clauses the sentence’s period (periodos) is made, that is the last closing-off of the thought, thus: “. . . and so they miss the traditional procedure of the courts.” But a sentence should not be longer than ... (possible) ...in one breath. (p75 (89 of 489))
(Note .1. Legal clauses, i.e. sections, are caput, clausula, or elogium.)
(Note .2. The only use I could find in Quintilian was e.g.'A fine turn of phrase.' Quint. prooem. § 22; 8 prooem. § 13; 8, 1, 1 et saepe But surely Quintilian must discuss this.)