Google’s SensorFM model turns wearable data into a universal health AI layer

Google Research has introduced SensorFM, a foundation model designed to transform the fragmented landscape of wearable health data into a unified, general-purpose AI system. Unlike today’s health features, which are typically siloed into single tasks like sleep tracking or cardiovascular risk assessment, SensorFM learns a shared representation of human physiology and behavior from continuous, often incomplete sensor streams. The model was trained on more than a trillion minutes of multimodal data collected from five million Fitbit and Pixel Watch users across over 100 countries, using more than 20 different device models. This dataset, the largest of its kind for wearable AI, enables SensorFM to handle diverse sensor inputs, including optical heart rate monitoring, acceleration, skin conductance, skin temperature, and barometric altitude, while accounting for missing or gappy data through a technique called Adaptive and Inherited Masking (AIM).

The foundation model’s strength lies in its self-supervised learning approach, where it reconstructs deliberately masked segments of data to uncover underlying patterns. This method reduces the need for expensive labeled datasets, a common bottleneck in health AI development. Google tested four model variants, ranging from 100,000 to 100 million parameters, and found that larger models paired with more data consistently improved performance. On the largest dataset, the biggest model reduced reconstruction error by 31% compared to the smallest variant. When evaluated on 35 distinct health and behavioral prediction tasks, spanning cardiovascular health, metabolic function, mental well-being, sleep analysis, demographics, and lifestyle factors, SensorFM’s learned representations outperformed supervised baselines with hand-crafted features on 34 of the 35 tasks. The model also demonstrated label efficiency, requiring fewer labeled examples to adapt to new tasks and relying less on demographic data as it scaled.

To further refine its capabilities, the researchers deployed a "classroom" of competing and collaborating large language model agents. These agents autonomously generated, tested, and refined code for downstream prediction models, running over 30,000 experiments in the process. The resulting models, built on SensorFM’s representations, outperformed simpler linear head models on 28 of the 35 tasks. The team also integrated SensorFM into a personal health agent prototype, comparing three variants: one that received SensorFM predictions, another that received actual known health data, and a baseline with no additional information. Four clinicians evaluated 93 health summaries for 31 real participant profiles, spending over 40 hours and producing 1,860 individual ratings. Summaries augmented with SensorFM predictions scored significantly higher than the baseline across all five measured dimensions, context, personalization, justifiability, relevance, and safety, while performing comparably to summaries using actual health data. However, the researchers emphasize that SensorFM is not a replacement for clinical measurements or diagnoses.

Despite its promising results, SensorFM faces several limitations. The model was trained and tested exclusively on data from Fitbit and Pixel Watch devices, leaving open questions about its performance on other wearables. It also operates on minute-level aggregated data rather than high-resolution raw signals, which may obscure fine-grained patterns. Many of the health markers studied were derived from self-reports, medication records, or questionnaires rather than clinically confirmed findings, and the study population did not fully represent the general population. Additionally, the health agent evaluation was conducted in a static setting with single responses, rather than in dynamic, conversational interactions.

For now, SensorFM remains a research project with no announced plans for integration into Google’s existing health products, such as the Gemini-based Google Health Coach. However, its potential to serve as a technical foundation for future AI-driven health assistants is clear. By providing a general-purpose layer for interpreting wearable data, SensorFM could reduce the need for task-specific models and enable more personalized, context-aware health insights. The model’s ability to handle diverse sensor inputs and adapt to new tasks with minimal labeled data makes it a significant step toward more scalable and efficient health AI systems.

The research team has shared more details in a Google Research blog post and an open-access paper on arXiv, inviting further exploration and collaboration. While the immediate practical applications are still uncertain, SensorFM represents a compelling vision for how AI can unify and make sense of the vast, messy streams of data generated by wearable devices.