Apple’s New ‘Enhanced Visual Search’ Feature Sparks Privacy Concerns

Apple’s New ‘Enhanced Visual Search’ Feature Sparks Privacy Concerns

Apple has quietly introduced a new feature called Enhanced Visual Search for Photos on iOS 18 and macOS Sequoia 15. While the feature adds powerful search capabilities, it has also raised eyebrows due to privacy concerns. Enabled by default, Enhanced Visual Search uses advanced technology to identify landmarks and points of interest in your photos.

What is Enhanced Visual Search?

Enhanced Visual Search allows your device to recognize landmarks or points of interest in your photos. Using a combination of on-device machine learning and Apple’s servers, the feature helps you search for photos of specific places with ease.

The toggle for this feature reads:
"Allow this device to privately match places in your photos with a global index maintained by Apple so you can search by almost any landmark or point of interest."

This sounds convenient, but the implementation has left some users feeling uneasy.

How It Works

Here’s how the feature operates:

  1. On-Device Analysis: Your iPhone or Mac first uses machine learning to identify regions of interest (ROI) in your photos.
  2. Embedding Creation: If a potential landmark is detected, the system creates a vector embedding of the image’s ROI.
  3. Server Query: The embedding is sent to Apple’s servers for comparison against a global index of landmarks.
  4. Anonymized Results: Apple uses homomorphic encryption, differential privacy, and an anonymization network to hide your device’s IP address before sending the query. Results are returned to your device in encrypted form, showing potential landmark names and metadata.

Privacy Concerns

Despite Apple’s use of advanced encryption and privacy-preserving techniques, the feature sends data back to Apple’s servers. This “phone home” behavior contradicts Apple’s long-standing claim: “What happens on your iPhone, stays on your iPhone.”

Developer Jeff Johnson, who first flagged the issue, criticized Apple’s decision to enable this feature by default without user consent. He pointed out that users should have the choice to opt in, particularly when privacy risks—however small—are involved.

How to Turn Off Enhanced Visual Search

If you’re uncomfortable with this feature, you can disable it:

  • On iPhone: Open the Settings app > Apps > Photos > Enhanced Visual Search and toggle it off.
  • On Mac: Open the Photos app > Settings > General > Enhanced Visual Search and uncheck the option.

Apple’s Response

Apple has defended the feature, emphasizing its use of privacy-preserving technologies like homomorphic encryption and differential privacy. A blog post titled “Combining Machine Learning and Homomorphic Encryption in the Apple Ecosystem” explains how the system anonymizes requests and keeps data secure. However, some users remain skeptical, particularly since the feature was enabled without explicit consent.

Balancing Innovation and Privacy

Enhanced Visual Search showcases Apple’s push toward smarter, more capable devices. But the lack of transparency and automatic activation has sparked valid concerns. Privacy-conscious users may feel that Apple’s approach undermines their trust, especially when the company has positioned itself as a privacy-first brand.

Final Thoughts

While Enhanced Visual Search is undoubtedly impressive, it highlights the tension between innovation and user privacy. For those who value absolute control over their data, turning off the feature may be the best choice. As Apple continues to evolve its technology, it’s crucial that it remains transparent and gives users more control over features that interact with sensitive data.

Back to blog