There are two main things to consider when testing smart lights:
- They need to function efficiently as a light (as well as have good luminosity)
- Their additional smart features need to be responsive and easy to use.
Our expert testers
CHOICE maintains a NATA-accredited laboratory and the vast majority of our product tests, including light bulb and smart home tests, are done in-house.
We have several years' experience in testing standard light bulbs and a sophisticated array of calibrated equipment for the task. We also have a long history of testing connected smart products, such as those belonging to the internet of things (IoT). This combination of lighting and IoT testing makes for a good fit when reviewing smart lights.
How we choose what we review
Smart lighting is still a relatively new category with few brands when compared to standard lighting. As such, it's possible for us to cover many of the major brands in the market.
We choose products that are readily available for purchase in Australia, while also looking at how prominently each product and brand appears in Australian web searches.
How we test
Light output
Initial average brightness is measured by placing the bulb into a calibrated integrating sphere, which is connected to a light meter that records the lumens output of the bulb at five-second intervals.
Each bulb is run for a minimum of two hours so that it can reach a period of stability across two consecutive 10-minute periods, and we average out the light output during this period of stability.
The lumens figure we obtain is then compared against the claim on the box as a percentage, and a scale is used to assign a score based on that percentage difference.
We make a point to test each bulb with its out-of-box, default hue, at maximum brightness, but note that the luminosity will be different when different colours are used.
Our measure of "initial" average brightness refers to each bulb being fresh out of the packaging. It's possible for bulbs to lose luminosity over time. We didn't test this for smart lights.
Energy efficiency and power consumption
We measure each light bulb's energy efficiency, rated in lumens per watt – the amount of light put out per watt of electricity used.
We also compare energy usage cost, estimated at 30 cents per kilowatt hour, assuming each bulb is active for five hours per day and in standby mode for the remaining 19.
Heat
Heat, measured in degrees Celsius using a thermal imaging camera, records the hottest part of the bulb (usually near the base) after a bulb has been active for more than an hour.
Set-up
We rate how easy the set-up process is for each product, taking into consideration installation, app design, connecting each light to Wi-Fi or its wireless bridge, allotting rooms, and creating an account online.
All LED smart bulbs are tested on a 2.4GHz Wi-Fi network with an ADSL2+ internet connection.
App layout, functions and responsiveness
We rate how easy each smart light app is to use, and app-responsiveness. We also record features such as sunrise/sunset controls, away mode and geofencing.
App responsiveness is evaluated in a same-room scenario (the router is in the same room as the bulbs), and we judge not only the responsiveness of the bulb turning on and off, but also its dimming and changing of colour.
Test criteria
The overall score is made up of:
- Ease of use score (60%)
- App responsiveness (25%)
- Luminosity (10%)
- Energy (5%)
The ease of use score is made up of:
- Set-up and documentation (45%)
- How easy it is to reset each bulb, for the purposes of installing the bulbs on another network or for troubleshooting (10%)
- The layout and functionality of the app through which these bulbs need to be controlled (25%)
- How easy they are to use remotely (20%)
Additional tests
We also conduct tests from 40 metres away (lights are installed 40 metres away from the router in a line-of-sight scenario), and from two non-line-of-sight scenarios.
These tests don't factor in our scoring. They are conducted in order to identify any performance issues across large areas.
We observed no issues with the performance of the lights in these tests.
Ready to buy?
Check out our latest smart light reviews and see our buying guide for information on the different types and features to look for.
Stock images: Getty, unless otherwise stated.
