There is a difference between sexuality (capacity for sexual feelings) and lust (very strong sexual desire).  

Not everyone experiences lust, but everyone experiences a capacity for sexual feelings.  

Strong physical attraction does not always translate to strong sexual desire as some may prefer to wait and see if anything meaningful can come from the attraction, have it develop into a relationship or leave it as a passing fancy.  

The reason why lust is a cardinal sin is that it goes beyond mere sexual attraction; it is the act of strong sexual desire without the meaningful component of respect or honor in the seeking of a meaningful relationship.

Lust is solely tied to the (sins of the) flesh; nothing spiritual or loving.  

When there are excessive practices into sins of the flesh or lust, the heart of the person lusting becomes unclean or dirty, because their mind automatically goes to the gutter in objectifying the object of their strong sexual desires; 

There is no honor and respect in treating a person as an object to be objectified and lusted after; therefore, lust of the flesh is sexual immorality.

Note:

1 John 2:16 says: For all that is in the world — the lust of the flesh, the lust of the eyes, and the pride of life — is not of the Father but is of the world.