Why do we use the nominative case in this example:
Liber legendus est. = The book needs to be read.
If liber is a direct object, then why not put it in accusative?
The gerundive is a passive entity: it (whatever it is)-ought-to-be-done. Here: "The book: it-ought-to-be-read". So, "book" is the subject and is therefore nominative, by definition; not "he read the book" in which case "book" would be in the accusative.
This confusion can arise in other areas: remember, "consul fieret"; was convinced that it should be "consulem"; no: "the consul he would become", not quite the same as "he is made consul".
It was clear from Qs, asked in April ("positive & negative", Joonas) that the gerundive can be something of a trial (me, with indirect speech). With any gerundive apply "it-ought-to-be"; or, "it-ought-not-to-be" then manipulate as appropriate.
The answers given by Tony, Kingshorsey, and Joonas are correct and should be enough for learners of Latin. However, it is true (and interesting!) that, from a linguistic/philological point of view, the apparently innocent question raised above by Imc ("If liber is a direct object, then why not put it in accusative?") is more complex if one considers, for example, what Woodcock (1959: 159) [A New Latin Syntax] says:
"In early Latin the impersonal construction is still found with the gerundive of transitive verbs, as Plaut. Trin. 869 Agitandum est vigilias 'One must keep watch'". (Cf. the Classical construction Agitandae sunt vigiliae).
So it seems that at least in Early Latin a transitive impersonal construction like Legendum est libros is not unexpected. This construction is said to be an archaism in Classical Latin: e.g., Cic. De Sen. viam quam nobis quoque ingrediendum sit 'the road which we, too, must tread' (cf. the more usual viam quae...ingredienda sit). Cf. also Lucr. 1, 111: aeternas quoniam poenas in morte timendum est ('Since one must fear after death eternal punishment').
NB: it is worth pointing out that not all Latinist scholars agree with Woodcock in analyzing agitandum as a gerundive in Plautus's example above. Some have claimed that it is a gerund. But if so, some obvious questions arise: e.g., (i) is there a nominative form for the gerund?, (ii) what about the clear modal meaning of agitandum above (cf. English transl. 'must'), which is typically absent from gerunds, etc.
Comparison to English might help here. (English is often misleading for Latin, but here it can at least illustrate the same phenomenon.) English distinguishes between nominative and accusative for personal pronouns, for example "he" vs. "him". Therefore the best analogue is found with pronouns instead of nouns.
Consider the sentence: Ille amandus est. Why is it not illum? Compare this with English. We say "he is to be loved", not "him is to be loved". The passive meaning comes from "loved" being a passive participle. The pronoun "he" is the subject of "is" and therefore in nominative.
This works the same way in Latin: the gerundive (or a perfect participle) is passive in meaning.
It is also worth noting that there is the object-free option legendum est, "one has to read". An object can't be added to this kind of construction in classical Latin; instead, you get nominatives like in your example. This relation is explored in this question.
The gerundive has no object; it functions as a passive adjective. It can be used attributively or predicately.
Liber legendus = a/the needing-to-be-read book
Liber legendus est = a/the book is needing-to-be-read