The average consumer can't tell Apples from Oranges

The Trademark Trial and Appeals Board ruled a couple weeks ago that the stylized orange above is too similar to the stylized apple above.
The fundamental principle of trademark law is identity. Infringement happens when there's a likelihood that people will confuse one for the other.
There are all sorts of other nuanced levels of where confusion lies, but when someone says something is like comparing apples to oranges, you can point to:
Apple Inc. v. Echospin, LLC
for the position that both are fruit and the average person can't tell the difference anyway.