I notice that in the De Naturis Animantium of Suetonius, he uses the genitive to describe the subjects of behavior. So, for example, he writes est [...] anatum tetrissitare ("it is of ducks to quack"). I get the sense of this, but is there a name for this use of the genitive. Why would he just not write, anas tetrissat ("the duck quacks") like we do in English?
c. An infinitive or a clause, when used as a noun, is often limited by a genitive in the predicate.
Neque suī iūdicī [erat] discernere. (B. C. 1.35)
Nor was it for his judgment to decide.
(Nor was it his judgment's to decide)
Cûiusvīs hominis est errāre. (Phil. 12.5)
It is any man's [liability] to err.
Negāvit mōris esse Graecōrum, ut in convīviō virōrum accumberent mulierēs. (Verr. 2.1.66)
He said it was not the custom of the Greeks for women to appear as guests (recline) at the banquets of men.
Sed timidī est optāre necem. (Ov. M. 4.115)
But it's the coward's part to wish for death.
Stultī erat spērāre, suādēre impudentis. (Phil. 2.23)
It was folly (the part of a fool) to hope, effrontery to urge.
Sapientis est pauca loquī
It is wise (the part of a wise man) to say little.
(Not sapiēns [n.] est, etc.)
Note 1: This construction is regular with adjectives of the 3rd declension instead of the neuter nominative (see the last two examples).
Note 2: A derivative or possessive adjective may be used for the genitive in this construction, and must be used for the genitive of a personal pronoun.
Mentīrī nōn est meum [not meī]
It is not for me to lie.
Hūmānum [for hominis] est errāre.
It is man's nature to err.
(To err is human)
Allen and Greenough classify this under possessive genitive (their §343) as do many other grammars.
Your translation "it is of ducks to quack" is good, and there is a difference between that and "ducks quack". This difference is the same in English and Latin, at least with decent accuracy. (The most natural English phrasing often takes a different route. The examples in A&G above give good ideas.)
The simple "ducks quack" or anates tetrissitant expresses a simple fact: these animals do this thing. There is no comment on the nature of this fact. They could be quacking just this once or it could be a more common thing.
The version "it is of ducks to quack" or anatum est tetrissitare expresses something more grave. As the possessive genitive suggests, the animals own this action: it is their characteristic habit, it is their duty, it is their tragic weakness, or some such thing. This certainly implies that ducks quack, but not only that.
The exact English translation depends on context. Here it could be "it is for ducks to quack", for example.
This use of the genitive doesn't seem to have a specific name. You can well call it "possessive genitive with infinitive" if a label helps organize things.
Joonas provided an excellent grammatical reference: A&G have worked out a very good classification of classical Latin idiom. I want to elaborate on the semantics of the possessive construction, possibly help you develop an intuition behind it, and make the answer to the second part of your question intuitive. We're lucky that a parallel construction with the possessive exists in English. It's not a reflex of the common PIE speech, rather an independent development, but using a possessive construction to express a sense of strong belonging is natural, and common across languages. Emphasis on a strong connection is the defining property of the possessive construction. It's nearly a tautology: the possessive emphasizes possession.
To understand the link, we only need to note that (1) the English possessive case in
's corresponds to the Latin genitive case, and (2) that the infinitive is a noun in nominative case¹ and a subject in the Latin syntax of such a construction. Then it simply reads “X is Y's,” where X being a nominal infinitive is an added twist: X=tetrissitare, Y's=anatum. But this is not exclusively Latin. Nominal infinitive exists in English, too.
We may need a digression into the subject of modern syntax, as such a use of the infinitive may be a stumbling block at first. You may be wondering: how could a verb form be a first class noun? But the classification of parts of speech from the syntactic point of view has nothing to do with word's meaning: [morpho]syntax is the analysis of certain language units (words, morphemes) which are devoid of meaning. It's a sad fact that the definition of a noun, still taught at school to this day is, in the end, circular: “a word used to identify any of a class of people, places, or things, or to name a particular one of these.” We can agree with this definition and break the cycle only by reinterpreting it as stemming in the word's syntactic role in a particular sentence, not an inherent property of a vocabulary lemma, as it's often incorrectly taught. A. Carney, who wrote an undergrad textbook on syntax (unsurprisingly titled Syntax) drills deep into the distinction in this short lecture.
Keeping this in mind, let's go through a progression of a few short English sentences that demonstrate both phenomena, and possibly create an intuitive connection between Latin and English possessives.
1. Possessive of ownership or inherent property
This is dad's house.
Nice and clear: dad's is a possessive qualifier of the object of the sentence, house. Both dad and house are clearly nouns, one a person, another a thing, and dad owns house. The possessive here expresses ownership of the thing by the person.
The coin is mine!
Note that this is a complete sentence, while an attributive my coin is only an NP. You cannot say *My coin! to claim that the coin belongs to you. Why not? My coin doesn't send a complete message to the listener, but the coin is mine does. You add information by putting emphasis on the claimed ownership of the coin.
The fact that a possessive forms a complete sentence doesn't have to do with syntax demanding the copular verb is or a special possessive mine. As a counterexample, there are languages that neither have a lexically distinct possessive pronoun nor use a copular verb in a similar construction, and express the same possessive semantics using word order alone. In a literal word-for-word translation, the attributive NP can be my coin and the full possessive sentence Coin my!
Quacking is the duck's trait.
Possessive here doesn't express ownership, but a property, with the emphasis on having the property. From the syntax perspective, note that the participle 'quacking' took the subject position, where only a NP is admitted. It's a syntactic noun, the head and the only constituent of the NP.
A periphrastic construction with the same meaning, but non-possessive syntax. It took almost twice the count of words to express the same statement.
2. Nominal infinitive
I fear hobgoblins.
Nice and clear: fear is a transitive verb, which takes a noun, hobgoblins, as the subject of the VP of which it's also a head. But, in
I fear to think what might have happened!
the same position can be comfortably occupied by the infinitive to think. We've already seen a participle in the noun's position, and this is another step in the same direction. Nominal infinitive exists in English, albeit only in the subject position.
3. Why not just "the duck quacks"?
By now, it should be clear that the infinitive is not at all special in this type of expression. The only thing of importance is that, although nominal infinitive exists in English, it's (no longer) used the subject position. While The coin is Bob's is A-OK, we do not say *To quack is the duck's.
Why would he just not write [...] "the duck quacks" like we do in English?
You already sense the answer: we don't! He could, but the meaning would be different. Alexander Pope's language is far from modern, and we no longer say “to err is human, to forgive, divine” (Criticism, 525), but if we did, I could be able to say “to quack is the duck's,” emphasizing the connection between quacking and ducks. You likely feel that a simple statement of fact “the duck quacks” (that duck or the duck in general? quacks now or quacks always?) lacks this precision and emphasis.
¹ When an oblique case is called for, the gerund is used instead, which—surprise, surprise!—lacks the nominative.