There's a whole range of expressions of the type (non) facio (or habeo, aestimo, etc.) + the genitive of some substantive, meaning 'I (don't) value as worth x.' (The genitive is a genitive of value.) The substantive can be something like parvi/magni, minimi/maximi, or tanti; or it can be some more concrete noun, like assis ('a small coin'), pili ('a hair'), or (my favorite) flocci ('a tuft of wool').
For example, from Plautus, Casina, lines 330–332 (Lysidamus speaking):
quid id refert tua?
unus tibi hic dum propitius sit Iuppiter,
tu istos minutos caue deos flocci feceris.
What does it matter to you? Provided that this Jupiter alone is favorable to you, see that you value those lesser gods as worth a tuft of wool.
In answer to your question, your translation, intersum sicut cuminum, doesn't work because the first-person form means 'I matter/make a difference,' not 'It matters/makes a difference to me.' You would have to use the third person interest instead. Then, to get a first person idea, you would add the feminine ablative singular of the possessive adjective: meā interest. (If the person/thing that something matters to isn't a personal pronoun, you would add the genitive of the noun – e.g., Ciceronis interest, 'It matters to Cicero.')
The third person of refero works the same way and means the same thing, as in the first line of the quotation from the Casina; so you can also say meā refert (or Ciceronis refert).
Most commonly, interest and refert are accompanied by adverbs (including adverbial accusatives) to express the extent to which something matters/makes a difference – e.g., nihil/nil/nihilum, parum, multum, permultum, magis, maximum/maxime, plus, plurimum, quantum, tantum, quantulum, nimium, pernimium, quid, nonnihil, aliquid, ecquid, numquid, minimum, or haud.
A genitive of value can also be used. However, in all the examples I found, the genitive was something like magni, permagni, parvi, quanti, or pluris, never a concrete noun. This is not to say that a concrete noun such as cumini can't be used (mea cumini interest, 'It makes a cumin seed's worth of difference to me'); as always, it just means that there are no examples like this in the extant classical literature.
So you could say something like one of these:
- nihilum mea interest/refert. 'I don't care at all.'
- haud mea interest/refert. 'I hardly care.'
- quam minimum/minimi mea interest/refert. 'I care as little as possible.'
- quantulum mea interest/refert! 'How little I care!'
- mea parum refert. 'I care little enough to matter.' (This is a quotation from M. Cornelius Fronto.)
- num mea refert? 'Do I care?' (The answer 'no' is implied here because of the num. This is Plautus, Truculentus 723.)
(To capture the effect of the English 'damn,' I suppose you could add an edepol or the like to any of the preceding – e.g., nil edepol mea interest, 'By Pollux, I don't care at all.')
The simplest answer, though, may be to focus on your second version of the Spanish phrase. One of the adverbs that can be used with interest/refert is tantum, and tantum and quantum form a correlative pair that's equivalent to Spanish tanto como. Therefore, you could just say (tantum) mea interest quantum cuminum, 'It matters as much to me as as a cumin seed (matters).'