Smartphone Compare: Objective vs. Self-Reported Smartphone Usage Analysis

Introduction

In today’s interconnected world, smartphones have become indispensable, with billions globally relying on them daily. This pervasiveness presents a unique opportunity to leverage objective smartphone data for real-world research across various disciplines. Indeed, the exploration of smartphone use in relation to personality, cognition, health, and behavior has surged in recent years. However, despite this growing interest, smartphones are not yet fully integrated into standard psychological research methodologies, and the validity of self-reported smartphone usage estimates remains largely unexamined. This raises a critical question in the realm of Smartphone Compare: how accurate are our perceptions of our own smartphone habits when pitted against objective data?

Miller [9] emphasizes the urgent need for social science researchers to stay abreast of advancements in smartphone research methodologies. A significant hurdle in adopting objective (actual) smartphone use data lies in the complexity of developing suitable applications and analytical tools capable of processing, analyzing, and visualizing large-scale datasets [10]. While open-source tools facilitate the creation of Android applications even for those without programming expertise [11], the open-source landscape for analyzing and visualizing the resulting data is still lacking. This gap underscores the importance of smartphone compare: to understand the strengths and limitations of different approaches to measuring smartphone usage.

Self-report methods have been valuable in contexts where objective data collection is challenging. However, their suitability for estimating smartphone use is questionable. While estimations might suffice for certain research questions, cognitive research on time perception suggests our inherent limitations in accurately estimating durations [12]. Subjective estimates are also prone to overlooking rapid, yet frequent, phone-checking behaviors [13]. Therefore, a robust smartphone compare analysis is crucial to determine when self-report data is adequate and when objective measures are necessary for valid conclusions about smartphone usage.

This study proposes that a straightforward measure—tracking when a smartphone is actively in use—can yield substantial insights into an individual’s daily routine. We aim to elucidate various metrics for objectively evaluating smartphone data and their implications for understanding smartphone use. To bridge the analytical gap, we provide open-source code for processing, visualizing, and analyzing objective smartphone data, making these tools accessible to researchers with varying programming skills. As a practical application of smartphone compare, we investigate the widely held notion of habitual smartphone checking behaviors. We correlate self-reported smartphone use estimates with actual smartphone use and established measures of problematic mobile phone use [14]. Ultimately, we aim to highlight the potential of objective smartphone data in psychological research and contribute to a more nuanced understanding of smartphone compare methodologies.

Method

Participants

Twenty-nine participants were recruited for this study, comprising 17 females and 12 males, with a mean age of 22.52 years (ranging from 18 to 33 years). All participants were Android smartphone users from the staff and student body of the University of Lincoln. The sample size was determined based on a priori power calculations to detect a moderate correlation between actual and self-reported use. The study protocol adhered to the Declaration of Helsinki guidelines, and all participants provided informed consent, both written and oral, after receiving detailed information about the study’s purpose and data collection procedures. Ethical approval was granted by the School of Psychology Research Ethics Committee at the University of Lincoln. Participants received a small compensation (£10) for their time. Data from two participants were excluded due to technical issues during the study, and four additional participants were excluded for incomplete self-report data. The final analysis included data from 23 participants, ensuring a rigorous smartphone compare between objective and subjective measures.

Materials

Smartphone Application: An Android smartphone application was specifically developed for this study using Funf in a Box [11]. This platform facilitates the creation of data-collecting apps for Android devices by allowing users to select sensors and define sampling frequencies. We selected the “screen on/off” sensor, resulting in a lightweight application designed to record timestamps marking the start and end of each phone use session. Data collected by the app was encrypted and securely uploaded to a server via Wi-Fi. Crucially, the app recorded a timestamp when the phone became active and another when the interaction ceased. While initially intended to measure screen state, testing revealed that the sensor detected any interactive state, including phone calls and music playback, even when the screen was off. This nuance is important for accurate smartphone compare interpretations.

Mobile Phone Problem Use Scale (MPPUS): The Mobile Phone Problem Use Scale (MPPUS) is a 27-item questionnaire known for its positive correlations with self-reported mobile phone use [14]. It remains a widely used instrument in health and psychological research [15–19] and has been employed as a broader measure of mobile phone use [6, 20, 21]. In our sample, the MPPUS demonstrated high internal consistency (Cronbach’s alpha = .89 for standardized items), reinforcing its reliability for smartphone compare analyses involving problematic usage tendencies.

Procedure

Upon arrival at the lab, participants had the custom-built smartphone application installed on their personal Android devices. To establish a baseline for message checking speed, participants were sent a standardized SMS message and asked to verbally relay it back to the experimenter. The time elapsed from the notification sound to the message relay was recorded. For the subsequent 14 consecutive days, participants were instructed to estimate their daily phone usage duration each evening, focusing on periods when the phone screen was active, aligning with the advertised function of the Funf on-off sensor. However, as noted, the sensor measured interactive state, not just screen activity. This discrepancy may have subtly influenced participants’ subsequent estimations, a factor to consider in our smartphone compare.

After the two-week data collection period, participants returned to the lab. They were then asked to provide a retrospective estimate of their average daily phone use duration (including calls and music) over the 14 days. This average daily estimate served as the primary subjective measure for smartphone compare against objective data. Participants were also asked to estimate their average daily phone use frequency (number of uses) and then completed the MPPUS questionnaire. Finally, the data-collecting application was uninstalled from their smartphones.

Data from the smartphone application were processed using Funf processing scripts to generate comma-separated values (CSV) files. These files were further analyzed using custom-developed source code to calculate descriptive statistics and create barcode visualizations of usage patterns, as depicted in Fig 1. The source code, detailed in S1 Appendix, allows for exploring usage metrics across different times of day (morning, afternoon, evening, and night) and analyzing checking behaviors of varying durations. These analyses could be performed for individual days or across the entire study period. Descriptive statistics were computed for the initial 14 days of the study. It was observed that some timestamps indicated exceptionally long single use durations (> 5 hours). This artifact was attributed to a limitation of the application: it did not reliably record screen-off events when the phone was powered down, leading to inflated “on” durations. To mitigate the impact of these outliers, median values were used for daily average use length calculations, providing a more robust measure for smartphone compare. Mean use length was then calculated for each participant based on these daily median values. The provided source code facilitates data visualization for individual participants across the study duration and can generate “average heatmaps” for daily, weekly, or weekday/weekend usage patterns.

Results

Objective Data

For each participant, we calculated the mean daily number of phone uses, the mean length of these uses (using daily median lengths to account for outliers), and the mean total daily duration of phone use. On average, participants used their smartphones 84.68 times per day (SD = 55.23) and spent 5.05 hours daily (SD = 2.73) engaged with their devices. As anticipated, the distribution of use durations was heavily skewed, with 55% of all uses lasting less than 30 seconds (see Fig 2). This skewness highlights the prevalence of brief interactions in overall smartphone usage patterns, a key aspect for smartphone compare against self-reported perceptions.

To further investigate short-duration “checking” behaviors, we defined checks as uses lasting up to 15 seconds. Analysis of these checks revealed three distinct peaks in usage duration: 1-3 seconds, 5-6 seconds, and 10-11 seconds. Fig 3 presents a histogram of these checks, binned in 0.5-second intervals. In-lab measurements showed that the average time to unlock a phone and read a short message was 8.42 seconds (SD = 1.53). Considering real-world distractions, the 10-11 second peak likely reflects the time taken for common checking activities like reading short messages, checking the time, or reviewing notifications. We also examined whether these short durations could be attributed to automatic screen timeouts. However, analysis of default display off times (meanLOCKED = 274.88s, SDLOCKED = 842.85s; meanUNLOCKED = 282.06s, SDUNLOCKED = 524.33s) indicated that these default settings did not explain the observed spikes in short-duration use, further emphasizing the intentional nature of these brief checks, a critical point in smartphone compare of usage types.

We also compared phone use across different times of day: night (00:00–06:00), morning (06:00–12:00), afternoon (12:00–18:00), and evening (18:00–24:00), as shown in Fig 4. For this temporal analysis, we used median duration length (the median time participants interacted with their phone before turning off the display) and total duration spent using the phone within each time window. Phone uses spanning multiple time periods were assigned to the period in which they originated.

Three separate one-way repeated measures ANOVAs (with Time of Day: morning, afternoon, evening, night as the within-subject factor) were conducted for total daily duration, use length, and number of uses. One participant was excluded from the total daily duration and median use length ANOVAs due to missing nighttime data. A significant effect of Time of Day was found for the number of phone uses (F(3, 78) = 34.62, p < .001, ηρ2 = .571). Post-hoc Tukey’s LSD comparisons revealed significantly more individual phone uses in the afternoon and evening compared to morning and night (all ps < .001), more uses in the morning than at night (p < .001), but no significant difference between afternoon and evening use frequency (p = .083). These patterns are visualized in Fig 4a. In contrast, no significant differences were observed across times of day for total daily duration (F(3, 78) = .94, p = .414, ηρ2 = .036; Fig 4b) or median use length (F(3, 78) = 2.33, p = .081, ηρ2 = .082; Fig 4c). These findings from objective data provide a detailed picture of smartphone usage patterns across the day, serving as a crucial benchmark for smartphone compare with subjective estimations.

Comparison of objective and subjective measures of smartphone use

Paired-samples t-tests and Pearson correlations were employed to directly compare actual and estimated smartphone use, as summarized in Table 1. For the number of phone uses, a significant discrepancy emerged: actual uses (84.68) were far greater than estimated uses (37.20; t(23) = 3.93, p < .001). Furthermore, no significant correlation was found between estimated and actual number of uses (r(21) = .11, p = .610), indicating a poor alignment between subjective perception and objective reality in terms of usage frequency. However, for total daily duration, no significant difference was found between actual (5.05 hours) and estimated use (4.12 hours; t(22) = 1.78, p = .086), and a moderate positive correlation was observed (p = .02). This suggests that while individuals may underestimate the frequency of their phone interactions, their estimations of total usage duration may possess reasonable relative validity. This contrasting result is central to the smartphone compare analysis, highlighting the variable accuracy of self-reports depending on the usage metric.

Finally, we explored the relationships between MPPUS scores, objective smartphone use measures (actual duration and number of uses), and estimated smartphone use using Pearson’s correlations (see Table 1). None of these correlations reached statistical significance (ps > .15). Notably, ten participants scored more than 2 standard deviations above Bianchi & Phillips’ [14] mean MPPUS score, indicating potential problematic phone use within our sample, though this was not directly reflected in correlations with usage measures in this smartphone compare.

Discussion

Previous research has linked estimated levels of smartphone use to various factors, including sleep patterns, interpersonal relationships, driving safety, and personality traits [5, 7, 22, 23]. Our findings, through a rigorous smartphone compare approach, suggest that self-reported estimates of total phone use duration exhibit a moderate relationship with actual behavior. Conversely, estimations of the number of phone checks showed no discernible relationship with actual usage frequency; in fact, actual uses were more than double the estimated number. While the possibility of a larger effect size being obscured by our sample size cannot be entirely ruled out, our results indicate that estimated use may be inadequate when high-resolution data are required, particularly for capturing checking behaviors. However, for research designs focused on overall usage duration, self-reported estimates might offer a reasonably valid proxy. This nuanced understanding of smartphone compare is crucial for selecting appropriate measurement methods in smartphone research.

The prevalence of short checking behaviors observed in our study aligns with findings from Oulasvirta and colleagues [24], who collected data in 2009. Despite the dramatic increase in smartphone ubiquity over the intervening six years, our data suggest that the frequency of checking behaviors has not necessarily escalated. Interestingly, individuals appear to have limited awareness of how frequently they check their phones. While Oulasvirta and colleagues proposed this notion in 2012, our study provides empirical evidence supporting the habitual nature of rapid mobile phone interactions [25]. While interactions under thirty seconds have previously been classified as ‘checking behaviors,’ our data refine this, suggesting that habitual, goal- and reward-based checking actions, such as checking time or notifications, are more likely to be under 15 seconds. This refined understanding of checking behavior duration is a valuable contribution of our smartphone compare analysis.

In our study, MPPUS scores showed no significant correlation with any measure of phone use, whether objective or estimated. The MPPUS is often used not only to assess problematic phone use but also as a general measure of phone usage. To evaluate its validity for this broader purpose, we correlated objective phone use with MPPUS scores. Our findings do not necessarily invalidate the MPPUS, but rather suggest that smartphone use is multifaceted [26], driven by diverse motivations, and increased use does not automatically equate to problematic use [27]. While it might seem intuitive to assume that prolonged phone use indicates problematic use, heavy users are not necessarily problem users. It is crucial for smartphone research to distinguish between heavy use and problematic use [8], and our smartphone compare highlights the importance of using diverse measures to capture these distinct constructs.

Objective smartphone usage data holds considerable potential for various applications. For instance, almost all participants in our study used their phones as alarm clocks, and most reported habitual phone use before sleep. These usage patterns offer a non-invasive proxy for sleep duration, potentially augmenting sleep diary data [28]. Furthermore, extending our analysis to consider usage patterns across different days of the week could reveal additional social and occupational implications [29]. The detailed insights from objective data, as demonstrated through smartphone compare, can significantly enhance our understanding of daily routines and behaviors.

Trull and Ebner-Priemer [9] and Miller [10] advocate for the expanded use of smartphone data as a research tool in psychology, yet objective smartphone data remains underutilized. Our study demonstrates the place of self-reported smartphone use estimates in current research, while emphasizing the limitations of their validity, particularly for detailed usage analysis. We advocate for complementing self-report measures with objective behavioral measurements. Furthermore, we provide a novel method for automatically sampling and visualizing smartphone use frequency through a simple background application and accessible source code. We hope that the methodologies presented in this paper will help overcome existing barriers to accessing smartphone data for psychological research and serve as a foundation for future advancements in the field, particularly in refining smartphone compare methodologies and applications.

Supporting Information

S1 Appendix. Source Code for analysing smartphone use data.

Source code, example screenprobe.csv data file, and README.txt for processing, visualising and analysing smartphone use data. csv2data.m converts ScreenProbe.csv to usable data, while barcode.m allows visualisations to be generated. descriptives.m generates descriptive statistics that can be used for quantitative analysis. Source code requires Matlab version 2014b or later, but does not require any specific toolboxes.

https://doi.org/10.1371/journal.pone.0139004.s001

(ZIP)

Author Contributions

Conceived and designed the experiments: SA HS DAE. Performed the experiments: HS. Analyzed the data: SA. Contributed reagents/materials/analysis tools: SA. Wrote the paper: SA DAE HS LP.

References

1. GSMA. The mobile economy 2014. London, UK2014.
2. Lane ND, Choudhury T, Georgiou T, Lehmann S, Rentfrow PJ, Torous J, et al. Future directions for mobile phone sensing in health. Pervasive and Mobile Computing. 2014;13(0):115–32.
3. Ehrenberg A, Jarry J, Jarry JL, Erdfelder E, Abraham C. Predicting internet use of British 16-24 year olds. Journal of Broadcasting & Electronic Media. 2008;52(4):563–79.
4. Roberts JA, Anderson LM, Crabtree CD. The predictive power of loneliness and social anxiety: Social networking site use and cell phone use among young adults. Personality and Individual Differences. 2014;69(0):250–7.
5. Augner C, Hacker GW. Associations between problematic mobile phone use and psychological parameters in young adults. International Journal of Public Health. 2012;57(2):437–41. doi: 10.1007/s00038-011-0340-z. PubMed PMID: 22045555.
6. Jenaro C, Flores N, Gómez-Becerra AI, González-Gil F, Caballo C. Problematic internet and mobile phone use: Association with psychopathological symptoms in Spanish teenagers. Cyberpsychology, Behavior, and Social Networking. 2007;10(6):913–6. doi: 10.1089/cpb.2007.9945. PubMed PMID: 18081415.
7. Thomée S, Härenstam A, Hagberg M. Mobile phone use and stress, sleep disturbances, and symptoms of depression among young adults-a prospective cohort study. BMC Public Health. 2011;11(1):66. doi: 10.1186/1471-2458-11-66. PubMed PMID: 21303561; PubMed Central PMCID: PMC3042898.
8. Bianchi A, Phillips JG. Psychological predictors of problem mobile phone use. Cyberpsychology & Behavior. 2005;8(1):39–51.
9. Miller G. Psychology in the cloud. Science. 2012;335(6075):1580–2. doi: 10.1126/science.335.6075.1580. PubMed PMID: 22442493.
10. Trull TJ, Ebner-Priemer UW. Ambulatory assessment in clinical psychology. Annual Review of Clinical Psychology. 2013;9(1):151–80. doi: 10.1146/annurev-clinpsy-050212-185559.
11. Aharony N, Pan W, Iacobelli F, Pentland AS. Funf journal: open mobile data collection and analysis platform. Open Mobile Data Workshop 2011. 2011.
12. Block RA, Zakay D. Prospective and retrospective duration judgments: an output-interference framework. Psychonomic Bulletin & Review. 1997;4(2):184–97.
13. de Jong T, de Valk L, Veling W, van Steenbergen H. The Check-Impulsivity Scale: Psychometric properties and associations with compulsive checking, impulsivity, and attentional control. Journal of Behavior Therapy and Experimental Psychiatry. 2013;44(4):465–72. doi: 10.1016/j.jbtep.2013.05.006. PubMed PMID: 23831354.
14. Bianchi A, Phillips JG. An investigation of cognitive predictors of problem mobile phone use. Behaviour & Information Technology. 2005;24(2):175–87.
15. Lin YH, Chang LR, Lee YH, Tseng WL, Kuo TBJ, Chen KH. Development and validation of the smartphone addiction inventory (SPAI). PLoS ONE. 2014;9(6):e98312. doi: 10.1371/journal.pone.0098312. PubMed PMID: 24914849; PubMed Central PMCID: PMC4051074.
16. López-Fernández O, Honrubia-Serrano ML, Freixa-Blanxart M, Bragulat-Barangué N, Alba-Rosell R, Molinuevo B. Prevalence of problematic mobile phone use in a Spanish adolescent population. Cyberpsychology, Behavior, and Social Networking. 2014;17(2):91–8. doi: 10.1089/cyber.2012.0604. PubMed PMID: 24354543.
17. Salehan M, Ramakrishnan S. Problematic mobile phone use in Malaysia: Antecedents and consequences. Computers in Human Behavior. 2014;41(0):222–31.
18. Tavolacci MP, Meyrignac G, Richard JB, Dechelotte P, Ladner J. Problematic use of mobile phone and symptoms of anxiety and depression among university students in France. BMC Public Health. 2013;13(1):660. doi: 10.1186/1471-2458-13-660. PubMed PMID: 23865869; PubMed Central PMCID: PMC3733748.
19. Yılmaz E, Griffiths MD, van der Linden M. Psychometric properties of the Problematic Mobile Phone Use Questionnaire-Short Form (PMPUQ-SF) in Turkish adolescents. Cyberpsychology, Behavior, and Social Networking. 2013;16(2):103–9. doi: 10.1089/cyber.2012.0108. PubMed PMID: 23384332.
20. Hong FY, Chiu TF, Ting SMH. An empirical study of internet addiction among Taiwanese adolescents. Cyberpsychology & Behavior. 2008;11(1):122–4.
21. Young KS. Internet addiction: The emergence of a new clinical disorder. Cyberpsychology & Behavior. 1998;1(3):237–44.
22. Demirci K, Akgönül M, Akpinar A. Relationship of smartphone use severity with sleep quality, depression, and anxiety in university students. Journal of Behavioral Addictions. 2015;4(2):85–92. doi: 10.1556/JBA.4.2015.1. PubMed PMID: 26114094; PubMed Central PMCID: PMC4495991.
23. Beran TN, Violato C, LeMay KR, Bayoumi I, Bhatnagar N, Kansal S, et al. Cell phone use, driving performance, and accidents: systematic review and meta-analysis. Traffic Injury Prevention. 2012;13(sup1):89–102. doi: 10.1080/15389588.2011.633455. PubMed PMID: 22435423.
24. Oulasvirta A, Rattenbury T, Anagnostopoulos A, Kumar E, Sarin R. Habits, routines and intentions: comparing methods for experience sampling of mobile phone usage. MobileHCI ’09 Proceedings of the 11th international conference on Human-computer interaction with mobile devices and services. 2009. p. 401–10.
25. Wood W, Neal DT. A new look at habits and the habit-goal interface. Psychological Review. 2007;114(4):843. PubMed PMID: 17924618.
26. Ling R. The social ubiquity of mobile communication: Investigating the domestication of SMS. Personal and Ubiquitous Computing. 2005;9(4):285–93.
27. Griffiths MD. Does internet addiction exist? Some philosophical and conceptual issues. Cyberpsychology & Behavior. 2000;3(2):211–53.
28. Lauderdale DS, Knutson KL, Yan LL, Rathouz PJ, Hulley SB, Liu K. Sleep duration: how well do self-reports agree with actigraphy? Epidemiology. 2008;19(6):838–45. doi: 10.1097/EDE.0b013e3181875a9a. PubMed PMID: 18849748; PubMed Central PMCID: PMC2737174.
29. Campbell SW, Kwak N, Flora J, Wang Y, Choi YJ. Microcoordination of mobile communication: Structuring time and space in everyday life. New Media & Society. 2010;12(2):195–211.