It does depend on the relative commonness of the two spellings, as you obviously suspect, jancsika.
For example, I accepted a request to allow
caddies, as it appeared that a golf caddy was much more frequently written as
caddy than
caddie. Moreover, a container for tea, etc, is always written as
caddy.
On the other hand,
when cabbies was raised, I found that the singular word was more often written as
cabbie than
cabby. I concluded that, "I think if I were to allow
cabbies, it would create more discontent than it would remove."
More recently I've been thinking that
any change I make affecting a plural word is likely to make things more confusing, so I've been deferring consideration of all such issues.