Sleep tracking theory

1. What is sleep

Simply put, sleep is something we do when we are tired in order to get un-tired. Human bodies are constructed in a way that they need a certain period of sleep after a certain period of activity.

We expect that when we wake up, we’ll feel better than when we went to bed.

But all sleep is not the same. Whether you have a healthy sleep and feel energetic or tired during the day is a result of many factors that go together or fight against each other. In order to understand why we sleep better or worse, we need to gain deeper understanding of the factors that affect sleep, and also be able to measure sleep objectively.

Sleep as Android enables you to track inputs and outcome of your sleep (the qualities of sleep itself).

1.1. Sleep input factors

The state in which you go to sleep, your bedroom environment, fall asleep and awake time, what you do during the day and of course your genetics are all input factors that determine how your sleep outcome is going to turn out.

You can do something about the majority of those input factors in order to steer the outcome towards a healthy sleep.

For that, Sleep as Android gives you tools such as:

1.2. Sleep outcome

Sleep outcome is the actual shape of your sleep - measured and analyzed. There are several ways to establish the quality of the sleep outcome, which together form a picture of your healthy your sleep is.

So when we want to know how well you sleep, we look at:

  • Sleep phases - especially the ratio of deep sleep versus light sleep

  • Duration of your sleep

  • Efficiency, ie. how long you’re actually sleeping vs. being in bed

  • Detected snoring duration

  • Your subjective quality rating

Those six dimensions together form your sleep score.

2. How is sleep tracked

Sleep scientists track sleep using two main methods:

2.1. PSG (polysomnography)

PSG is the first method used to measure sleep. If you go to a sleep lab, you will with 99% certainty undergo PSG.

This is the 'heavyweight' method - the patient is usually wearing a lots of sensors:

  • EEG (to measure brain activity),

  • EOG (to measure eye muscle movements),

  • Actigraphs (to measure overall movement),

  • EKG (to measure heart activity),

  • Pulse oximeter (to measure blood oxygenation level),

  • EMG (to measure other muscle movements),

  • and possibly other sensors.

PSG is regarded as a golden standard in sleep science as it measures the most sleep dimensions.

PSG distinguishes REM sleep and non-REM sleep. Non-REM is further divided into N1, N2, N3 (and previously N4) phases. Sleep phases are largely based on input from EEG. So four major phases characterized by specific EEG patterns are usually recognized today.

N1 (non-REM-1)

a short transitional stage between sleep and wakefulness.

N2

mostly light dreamless sleep occupying about 60% of the night.

N3

the deepest sleep phase. The body is totally relaxed, EEG displays slow regular waves. This stage is believed to play a crucial role in the regenerative processes. Dreaming (so called non-REM dreaming) can occur but is not that common as in REM.

REM

high likelihood of vivid dreams, muscle paralysis, bursts of rapid eye movements.

The phases alternate in a typical sequence called sleep cycle – from a short N1, via N2, to the deep N3, then a shorter transitory N2, followed by REM, interrupted by brief awake. The cycle repeats several times throughout the night. The pattern is however highly variable. The lengths and exact ordering of the phases differ in each individual case.

The biggest criticism of PSG is that it is too invasive - the patient is not in his home environment and is entangled in lots of wires and other electronics that prevent him from sleeping naturally. This complicates diagnostics.

2.2. Actigraphy

There is, obviously, no EEG in a smartphone or common wearables. However, we can monitor a sleeper with sensors that are available in these devices, and it makes perfect sense to analyze the measurements and see if they display any patterns. In Sleep as Android, we focus mostly on detailed monitoring of body movement throughout the night, using a wide range of available sensors (integrated accelerometer, sonar, infrared sensor).

And indeed, there are typically distinct phases of very low physical activity, when the body is completely relaxed, and periods of high activity, when the person is tossing and turning on the bed, rolling, twitching, and jerking.

Thereby in actigraphy, we can define two sleep phases – light sleep and deep sleep. These phases are shown in hypnograms in Sleep as Android and are used by the app for triggering smart alarms. Let’s call them ACT-phases, as they are based purely on the average short-term amount of physical activity. It’s an objective phenomenon, just like the PSG-phases. We can imagine them as a picture of the same underlying process (sleep) from a different angle. Either picture (PSG-phases, ACT-phases) captures a valid aspect of the reality.

2.3. Comparison of PSG to actigraphy

Actigraphy uses a different input than PSG and defines sleep phases in its own way.

Both ACT-phases (from actigraphy) and PSG-phases reflect an objective aspect of sleep. One naturally needs to ask if there’s any relationship between PSG-phases and ACT-phases.

2.3.1. Validation of actigraphy done by other teams

Several research teams raised similar questions before (See this one, or this, or this, or this). They measured a bunch of people on a traditional polysomnograph and recorded their physical activity at the same time. (By filming them and then counting the movements manually, or by using accelerometer readings). The published analyses show that there indeed is a significant statistical relationship between PSG-phases and body movements.

2.3.2. Validation of actigraphy done by us

We have measured how accurate Sleep as Android is when compared to a clinical sleep lab and concluded that light and deep sleep measured by our app strongly correspond with sleep phases measured on PSG. Read an extensive review by our data expert Jan Marek.

2.3.3. PSG-phases vs ACT-phases

Deep sleep ACT-phases detected by our app roughly correspond to N3 and partly N2 PSG-phases.
Light sleep corresponds to REM, N1, and partly N2.

PSG-phase Corresponding movement activity (ACT-phase)

N3

Deep sleep
The least amount of physical activity.

N2

Deep sleep or light sleep
The sleeper spends about half of the night in this phase. Movement intensity is somewhere in between the other phases. Sometimes the physical activity is low, especially when N2 precedes a deep sleep phase, sometimes the activity is higher, before or after N1 or REM phases.

N1 and REM

Light sleep, REM
The body is moving the most. The activity in REM might look surprising at first glance. It is known that large muscles are paralyzed during REM, as vivid dreams typically occur at this stage, and the body must be prevented from acting them out, jumping over the bed, riding a spaceship, fighting space villains. The paralysis is not perfect though, and limbs or trunk often randomly jerk and twitch.

However, the amount of movements varies greatly, based on age, gender, individual specifics, health, mental state, etc. There is no exact correspondence between such and such movement frequency and a specific sleep phase. The only overall reliable principle is that relatively low activity intervals are mostly N3 or N2, and relatively high activity intervals are likely to be REM, N1, or N2. Any attempt to pinpoint the exact beginning of a sleep phase is subject to a high degree of error and guesswork.

Nonetheless, this is still a useful approach, providing valuable insights with home-made sleep recordings. Large data can be collected cheaply for population-wise studies. Individual sleep enthusiasts may discover their own sleep patterns and possibly devise their own personalized sleep phenomenology.

FAQ

Added awakes by mistake

  • It is possible to revert up to 5 changes done during editing a graph.

  • You can use Undo banner (appears each time you change a graph):

awake mistake
Figure 1. Undo banner after using scissors icon
  • You can use ⋮ menu → Undo option

Duration is shorter, lenght is wrong

​The Sleep Duration is total sum of all your sleep phases (Light, REM, and Deep), not counting the awake phases - because when awake, you are actually not sleeping.
​So on default settings, the Sleep duration is always a bit shorter than the duration of tracking.

​If you wish your Sleep duration is the same as tracking duration:

  • Disable the awake detection in Settings → Sleep tracking → Awake detection.

  • You can also try to adjust the sensitivity of each type of settings to get optimal results. In most cases, too much awake periods are caused by significant HR peaks.

  • If you are not sure, where those awake periods come from, please use Leftmenu → Report a bug, and send us the application log.

How do I get BT smart heart rate device work with Sleep as Android?

  1. Enable the tracking in Settings → Wearables →Bluetooth Smart (might be hidden under Advanced section).

  2. Try to pair with your device (this may not be required for all devices and OS versions).

  3. Make sure no other app is using your device while sleep tracking.

If nothing helps please send us a debug report using Leftmenu → ic bug Report a bug.

Note BT Smart heart rate tracking only works from Android 4.3 onward

How does Sleep as Android (actigraphy) compare to Polysomnography?

We use a different input than polysomnographists, and define our own sleep phases, reflecting an objective aspect of sleep, easy to measure with common devices. One naturally needs to ask whether there is any relationship between the EEG-phases and our ACT-phases.

Fortunately, several research teams raised similar questions before (See this one, or this one, or this one, or this one). They measured a bunch of people on a traditional polysomnograph and recorded their physical activity at the same time (By filming them and then counting the movements manually, or by using accelerometer readings). The published analyses show that there indeed is a significant statistical relationship between EEG-phases and body movements.

You can also read about comparison of Sleep as Android algorithms and Sleep lab results on our blog here.

How does my phone know, when I am in the deep sleep?

We use a method called sleep actigraphy and there are several scientific papers which show that it is not as precise as PSG but still provides reasonable data and more over it is more convenient for the user. The phone has an accelerometer sensor build in which is very sensitive and if placed in your bed we have a record of your movement over the nigh. In deep sleep your muscular movements are suppressed (otherwise you would be running or jumping around according to your dreams :)) and thus in this phase the sleep graph gets nearly flat. This is in short how we measure your sleep phases, for more details refer to How it works.

How does the Battery saving mode in Sleep tracking work?

Battery saving mode currently resumes full tracking before the smart wake up period in order to find the best moment for your wake up, so the tracking uses up just a fraction of the battery consumption for the whole night. If the battery would drop under your defined stand-by threshold (default: 10%) the battery saving mode will re-occur.

I do not trust the results, it is fake / generating random data

Accelerometric sensors are really sensitive, which is great for sleep tracking. Normally, what you see when you leave the phone on the table gets immediately dwarfed when you do some more significant move. Just leave phone on the table for a while and you will see a dramatic development, but then move the phone and you will see all the development is really tiny in comparison to the new peak.

So what you see is random noise, given by very small vibrations of the table or in very calm areas by seismic movement. We mark the data relatively, so you always get it distinguished into light and deep sleep. But the algorithm works well only in conditions that are assumed by it, i.e. in the bed with relatively large movement peaks.
To be more specific, if you leave the phone on a table, you can get values perhaps on the scale of 0.000001 to 0.000009 m/s2 (The value is made up here, but it is physically very small). In the bed, you may get values from 1 to 9 m/s2 (which is physically large). The algorithm sees though just that the high value is 9 times higher than the low value, in both cases.
We had to do this because every accelerometer (in different cell phones) measures differently, so we couldn’t assume any standard conversion formula that would respond to absolute values.

So if you use the phone in the bed, it is in fact drastically different from measuring on a calm spot, just like the table.

Please do not hesitate to ask for any clarification at support@urbandroid.org.

I have a dog / cat sleeping with me in bed. Will the sleep tracking be accurate?

This depends on several factors. The general rule is to not allow the pet to move your phone, ideally only your movements should move the device. So in this case it’s best to place Your device either under the pillow or to have an armband or smartwatch/smartband. If your pet is a calm one, it may just work. However, if your pet is used to jump in and out of bed several times a night, the sleep tracking will most probably register these events as light sleep occurrences.

Is sonar safe?

Is sleep tracking with sonar safe for your health?

Ultrasound is generally considered safe if it is at normal volume. Regarding health effects, it works in a similar way to normal audible sound, i.e. very loud ultrasound can damage your hearing, whereas at low volume it is safe to hear. When using speakers, smartphones are nowhere close to be able to produce such loud sounds as to damage your hearing.
We also use ultrasound that is very close to the hearing range (around 20 kHz), so the effects of the ultrasound are almost identical to hearing a high pitched sound at the same volume (expect you can’t hear it at all).
The ultrasound volume we use is around 40 dB – which is lower than normal speech volume. You can measure the sound level yourself using e.g. this app.

Is sonar safe for your pets (cats, dogs, bats)?

For pets that are able to hear it, the ultrasound emitted from Sleep as Android is a constant low noise. The situation is similar to e.g. refrigerator noise. It is there, you can hear it, but it’s not so much disturbing. The ultrasound definitely cannot damage your pets hearing at the volume used in Sleep as Android.
Bats can be confused and fly into walls.

Is sleep tracking with sonar safe for your smartphone?

The only difference between normal audible sound and our sonar is that the frequency is a little higher (normal frequencies 2 Hz-20 kHz, our sonar frequencies 18 kHz-22 kHz). This is so small difference for the mic and speaker membranes that there is definitely no chance of damage, even with prolonged usage.

My graphs are flat

There can be several reasons why your graphs are flat.

The actigraphy is almost flat

When you can see some movement on the actigraphy, but the graph is unusually flat:
. Sonar - make sure the signal is strong enough by keeping sonar volume at max at Settings → Sleep tracking → Test sensor → blue sliding bar.
- you can also try different frequency by choosing other frequency from the ddrop down menu list in Settings → Sleep tracking → Test sensor → Frequency
- keep the phone closer to you bed
- try different positioning of the phone
. Accelerometer - try keeping the phone closer to you.

The actigraphy is flat, or red
  • Disable all system restrictions applied to Sleep as Android, or any companion app for tracking with a wearable: https://dontkillmyapp.com/

Phone gets hot during tracking

Usually this is not caused by the sleep tracking directly as this is usually not consuming too much resources (usually around 1-3% battery per hour of tracking).
The issue appears because we hold a wake lock (keeping the phone awake) – any badly written apps may access the CPU extensively during the sleep tracking time. We suggest checking which services are running before you get to sleep.
For us it is hard to debug this. Also battery statistics are not a hint here as all battery consumption is accounted to the app which holds the lock even it did not consume the battery – this is by design in Android.

To conclude, this issue may happen, although we did not get any similar reports for a very long time now. But the most probable cause is some wrong 3rd party service or app on your device.

To see more on the issue we would need a debug report (menu > report a bug).

A good test would be to reboot your phone before sleep tracking (or kill any unnecessary services running) and see if that helps.

Quick guide to start with Sleep as Android

At first Sleep as Android may look complex with all its options, but in fact it is fairly easy to get started improving your sleep and wake up.
You can delve deep into all the amazing options Sleep as Android provides later on. But to get started just rely on our carefully selected defaults.

  1. Use the top right plus icon to set up your Alarm time (e.g. 8:00). Tap the desired time directly on the clock - hours first, minutes second.

    quick alarm 1
  2. Confirm the selected time with SET button, it will show you the new alarm settings. Confirm the alarm with DONE.

    quick alarm 2
  3. Smart wake up makes sure to find the best time for your wake up between 7:30 – 8:00 based on your sleep cycle.

  4. You can now see the alarm scheduled on the main screen.

  5. Tap the moon and Sleep tracking starts, we use your phone’s sensor to find out what sleep phase you are in (see How it works).

    quick alarm 3
    Note We recommend turning on airplane mode during sleep tracking
  6. Now just place the phone on the mattress near your body (see Setup sleep tracking) so it is able to sense your movement and wake you in the right sleep phase to bring your wake up experience to a new level

  7. After dismissing the alarm in the morning you will see your sleep phases, deep sleep % and more..

Samsung Galaxy Gear - Watch app stuck on "Start tracking"

This can be a result of multiple things, so please make sure to do the following troubleshooting:

  1. Make sure you have Sleep as Android Gear Addon installed on your phone

  2. It can happen that the addon cannot be started by us if it was force stopped previously. In that case please go to Play Store app on your phone, open addon page tap on “OPEN”.

  3. Opt out of any battery savers that you might have on your phone, for all involved apps (Sleep as Android, Sleep as Android Gear Addon, Samsung Accessory Services) – to find out how to do that, please consult dontkillmyapp.com

  4. Samsung Accessory services sometimes misbehaves and prevents connection to the watch for 3rd party apps. Please uninstall and reinstall it.

Sonar is audible, strange sounds when using sonar

We have reports that on some device you can hear audible artifacts during sonar tracking. It sounds like this:

Some of the signal gets into audible spectrum probably due to either insufficient quality of the speaker or some post processing which is applied to the output on your device firmware.

We have also some reports that Sonar can get audible suddenly during tracking in the night. Unfortunately we are not sure why this could happen, we only have very few such reports and we are not able to reproduce this on our phones.

To make any audible artifacts less likely:

  1. Go to Settings > Sleep tracking > Test sensor.

  2. Try different frequency from the drop down menu list.

  3. When you find the least affected frequency, you could try lowering the volume a bit (the sliding bar). But keep it as high as possible to maintain reliable results.

Note If the volume needs to be adjusted, always confirm that sonar is still working - ideally after you change settings, try to sit calm in font of the test for few seconds and than move slightly - do you see a spike?

Too much awakes (false-positive)

  • When there is too much awakes falsely estimated on your graph, use Left ☰ _menu → ic bug Report a bug, and send us the application log.

  • Most often the awakes are driven by significant HR peaks (awakes align with HR red line graph), you can try disable this type of awake detection in Settings → Sleep tracking → Awake detection → Heart rate monitoring.

  • Other common reason is phone screen turned on, you can try to disblae Awake when using phone awake detection in Settings → Sleep tracking → Awake detection → Awake when using phone.

Tracking crashes, stops suddenly

If the tracking stops completely after few minutes, the background processes are restricted by your system.

  • Make sure no system restrictions are applied to Sleep as Android, or any companion app for a tracking with wearable: Check our guide here.

  • If the guide won’t help, send us your log using Leftmenu → ic bug Report a bug.

Tracking starts on its own

  1. Please make sure that you are not accidentally starting the Sleep as Android app from your watch. This would start sleep tracking immediately.

  2. Make sure you are not using automatic start of sleep tracking in Settings → Sleep tracking → Start sleep tracking.
    You can find more information about automatic sleep tracking start here.

Volume goes down when tracking

If your volume gradually lowers after a while, when the tracking is running, the option Turn off when sleeping is enabled. This feature lowers the system-wide volume, so it affects even music played outside Sleep.

  1. Go to Settings → Lullabies → Turn off when sleeping.

  2. Extend the minimal playback time (5-90 min).

  3. Or disable the feature completely by setting it to Use current device volume.

Volume jumps to max when tracking

Warning The volume needs to be kept at maximum when tracking with sonar for maintaining the reliable results.
  • Unfortunately, this also affects media volume in 3rd party apps, and we cannot control those separately from sonar media volume. This means that while using sonar, you can only use media apps on full volume.

  • You can set a time delay on start of tracking in Settings → Sleep tracking → Awake detection → Delayed sleep tracking.

External players
  • When using sonar, you cannot control media volume by volume buttons as it always jumps back to maximum.

Lullabies
  • You can control volume of lullabies from the Sleep app (Settings → Lullabies), and from the Lullaby add-on pack.

  • When you lower the volume with volume buttons, the lullaby volume is estimated and adjusted accordingly, sonar volume is still kept at maximum.

Why is Sleep eating so much battery? What about battery overheating?

Usually battery consumption issue or related issues causing phone over-heating during sleep tracking are not caused by the sleep tracking directly.

In most cases sleep tracking itself is not consuming too much battery (usually around 1-2% per hour of tracking). But because we hold a wake lock (keeping the phone awake) any other usually badly written apps may access the CPU extensively during the sleep tracking time. We would suggest checking which services are running before you get to sleep. For us it is hard to debug this. Also battery statistics are not a hint here as all battery consumption is accounted to the app which holds the wake lock even it did not consume the battery (this is by design in Android).
A good test would be to reboot your phone before sleep tracking (or kill any unnecessary services running) and see if sleep tracking will still consume too much battery afterward. Features within Sleep as Android which may cause higher CPU load during tracking include noise recording. You may try tracking without it for a reference. Also I would strongly recommend to track with airplane mode on.

Why is there a red bar / section / block in my sleep graph?

The red block indicates that something went wrong with tracking at that time and the device stopped providing sensor data for some reason. Usually those are some non-standard battery optimizations or battery savers, the battery gets too low so we preserve it for the alarm or connectivity issue if you use a wearable.

1. Battery restrictions

Make sure no system restrictions are applied to Sleep, or any involved apps like wearable companion app).
See our guide here, and follow the instructions.

2. Too low battery

When the battery is too low (usually below 10%), data collecting is terminated to preserve enough battery for alarm.
When the battery was too low, there is a battery icon is displayed on the graph:

low battery
Figure 1. Low battery graph

3. Connectivity issues with a wearable

When the connection with the wearable is lost, you can see red sections on the graph. The app always tries to reach the wearable again.
The graph can look like this:

red wearable
Figure 2. Connection lost during tracking
  1. Opt-out from any battery restrictions is applied by your system (https://dontkillmyapp.com/)

  2. Pair the wearable with your phone in System settings.

  3. Make sure the BT is not lost, and try lowering the distance between the phone and the wearable.

  4. Try settings the device as Trusted device.

Why it is not possible to enable airplane mode automatically after sleep tracking starts?

Unfortunately due to dummy security restrictions the Android team introduced in 4.2 there is no option to enable airplane mode from an app automatically. You always have to use the settings or long touch power button. If you have a rooted phone you may consider using https://play.google.com/store/apps/details?id=lv.id.dm.airplanemh we have support for that in Sleep. There is a similar hack for 4.3.

If you don’t agree with the Android team design decision you can upvote issue 40497 here http://code.google.com/p/android/issues/detail?id=40497.

Why sleep record data count towards the end date

There is no clearcut answer to which day the sleep between them belongs.

We have decided to attach the sleep to the day after, because how you slept will largely determine how your day will be.

Will sleep tracking work with two people in the bed?

If you have separate mattresses there is minimum interference from your partner. If you have one big shared mattress (which isn’t recommended as you partner may need different mattress for his healthy sleep), it could still work assuming you keep your phone close to your body and ideally on your side of the bed.

You can also consider using a armbands or smartwatches. This certainly solves the problem for a little convenience trade-off.

If both of you are tracking, you can enable pair tracking, which filters out the partner’s activity.