The molecule size difference is a non-starter.
O2 molecule is 292 picometers
N2 molecule is 300 picometers
At normal tire pressures N2 and O2 have almost exactly the same expansion ratio. At extreme pressures there's more of a gap, but we're not dealing with that here.
So tires with N2 have 2 insignificant advantages.
The inside of the tires aren't exposed to oxygen, so they will break down slower than ones with air in them. We don't see corroded tires though.
Compressed air has very little moisture in it, but it does have a little. At some extreme tire temps this might change the tire pressure, but at those temps that would be the least of your troubles.