Need to know
- We asked ChatGPT and Meta AI to recommend products to buy based on a few basic queries
- Their suggestions included items we've found to be poor-performers or unsafe
- CHOICE thoroughly and independently tests every product we review, so our verdict is (a lot) more reliable than AI
In the two years that most of us have had easy access to them, AI (Artificial Intelligence) chatbots have captured attention with their ability to simplify everyday tasks.
ChatGPT, launched in 2022, has been followed by a flurry of competing services from rivals such as Meta and Google.
All deliver quick, succinct and personalised responses to queries and are expected to play an expanding role in our lives, especially in areas such as customer service.
But how reliable is the information and advice these platforms put out?
Having already revealed that these services aren't very good at providing financial guidance, we've now quizzed them for recommendations on the best products to buy for around the home.
On this page:
- 5 poor-performing products recommended by AI
- Other issues with AI recommendations
- Why AI isn't good at recommending products
- CHOICE reviews help consumers in age of misinformation
5 poor-performing products recommended by AI
We quizzed the free versions of two popular AI chatbots – ChatGPT and Meta AI – for recommendations across a broad range of household categories.
For each of these categories, which included robot vacuums, air fryers and portable cots, we asked each chatbot to suggest five good products to buy in Australia.
AI is changing all the time and the following results are drawn from recommendations these chatbots gave us in October 2024. Here are five of their most dubious suggestions:
1. A robot vacuum bad at picking up pet hair and cleaning carpets
In a warning to anyone hoping ChatGPT could help them find a way to clean up after their pooch, one robo vac "highly recommended" to us by the chatbot is actually unlikely to impress in a hairy situation.
Samsung's Jet Bot AI+ was recommended by ChatGPT, but failed to impress in our tests.
ChatGPT claims the Samsung Jet Bot AI+ has strong suction power, implying it's good at picking up debris. But our real-life test results tell a different story.
CHOICE experts evaluate robot vacuums on how good they are at removing real pet hair from a real carpet and this $1699 RRP model came back with a score of 25% – equal second-worst in our latest review.
And its carpet cleaning score, which measures how good it is at removing dirt, was almost as poor – just 33%.
Considering we see cheaper robot vacuums performing better on both metrics, we won't be joining ChatGPT in singing the praises of this particular sucker. To see which models we do recommend, check out our latest robot vacuum review.
2. An air fryer that doesn't do a good job on chips
It's the primary use of more than a few home ovens, so you'd think an air fryer would be decent at heating up the humble frozen chip.
ChatGPT also suggested an air fryer outperformed by cheaper models.
Our air fryer reviewers check how good each model is at cooking popular foods and while the Ninja AF160 holds its own on some dishes, it falls short on frozen chips.
It scored just 60% when we ran it through its paces with 700g of straight-cut fries – the third-worst of all 37 models in our latest test.
The Ninja air fryer's poor potato performance makes ChatGPT's crowing about its "great crispiness" all the more bewildering.
Beyond just chips, we found we had to turn on the device's max crisp function to get good results on most frozen foods.
This $150 appliance didn't come last on overall performance, but its mid-range status and the fact that better air fryers are available for less means we find ChatGPT's recommendation questionable. Get our recommendations from our air fryer review.
3. A video doorbell with poor sound quality
Hello? Hello? What did you say? You might find yourself saying this more with the Arlo Essential Video Doorbell.
Also recommended to us by ChatGPT in response to our query "What are five good smart doorbells to buy in Australia?", this device came middle-of-the-pack overall when we tested the wire-free version, but anyone using it could be hard-pressed to hear the person speaking on the other end.
Its sound quality score of 40% put it among the bottom five on this metric in our latest review of 18 models.
It was also one of the more difficult devices to install, with instructions that failed to mention you might have to go online to get support to finish the job.
4. A bassinet that is potentially dangerous for babies
The Stokke Sleepi Mini V3 was one of Meta AI's top picks when we asked the Facebook-affiliated chatbot to recommend bassinets. It's an endorsement we would never give.
The Stokke was one of several models we found could pose a suffocation risk to babies and it therefore failed to pass our in-house safety test.
When placed in the bassinet, the required mattress is insufficiently firm at areas where the covering is puffed at the sides and ends – a suffocation risk if a baby rolls face first into them.
It's worth noting that at the time of testing last year, there was no Australian standard for bassinets, but there is one being phased in now. We used our own in-house method that considers whether the bassinet meets what we consider to be key safety requirements.
Following our verdict, Stokke had the mattress retested by an accredited third-party lab, where the sample passed. CHOICE stands by its results on the model we tested.
5. A portable cot that poses a suffocation risk
When we asked Meta AI to suggest some good portable cots, it again recommended a product CHOICE expert testing has found to be potentially dangerous.
It told us to consider Kmart Anko's 3-In-1 Portacot, but this model recorded a serious safety failure in our latest review.
Meta AI recommended a portable cot we've found to be potentially dangerous.
We found its mattress wasn't firm enough, creating a possible suffocation risk if a baby rolls onto their side or front.
It's worth noting that our safety tests measure portable cots against the requirements of the voluntary safety standard, which goes above mandatory requirements.
Kmart disagreed with our results, showing us reports from other third-party labs, indicating that their products pass safety standards.
CHOICE operates its own in-house accredited lab and stands by its results. To see which sleeping solutions we do recommend, see our latest portable cot review.
Other issues with AI recommendations
Beyond dodgy product suggestions, we found AI chatbots have several other flaws that make them less than ideal for helping you compile your next homewares shopping list.
These include a tendency to recommend brands in general instead of specific products and, if a prompt isn't specific enough, naming models that aren't widely available in Australia.
Why AI isn't good at recommending products
The AI chatbots we consulted in this investigation did occasionally surface good products.
But the fact that they can just as quickly recommend poor performers makes it hard to trust these platforms when settling on what could be a significant purchase.
While ChatGPT and Meta AI do have disclaimers admitting they make mistakes and advising users to verify information themselves, their increasing ubiquity means they could be pushing consumers toward products destined to be unsatisfactory and even dangerous.
So why is AI spruiking these items? To find out, we went straight to the source and asked ChatGPT and Meta AI where they get their recommendations from.
Its sources for this information included a product's own manufacturer, social media, internet forums and some for-profit comparison sites
ChatGPT explained it looks at performance, customer reviews, brand reputation and safety features when recommending products like robot vacuums, air fryers and smart doorbells.
However, its sources for this information included a product's own manufacturer, social media, internet forums and some for-profit comparison sites.
Meta AI, meanwhile, indicated it was solely relying on retailer's own websites when we asked it how it recommended bassinets and portable cots.
In response to questions from CHOICE about the problems we found with their platforms' recommendations, Meta didn't comment beyond drawing attention to its disclaimer. Open AI didn't respond to requests for comment.
CHOICE reviews help consumers in age of misinformation
CHOICE senior campaigns and policy adviser Rafi Alam says the penetration of AI as a relatively unregulated and experimental information source into our lives could cause trouble.
"Consumers are facing new risks as AI platforms grow in popularity, ranging from minor inconveniences to serious harms," he says.
"Fortunately, the government has started to look at ways to address these risks, including reviewing the law to see if it can be modernised to include mandatory guardrails on the use of high-risk AI across our economy."
CHOICE Expert Ratings, recommendations and reviews are the result of extensive independent product testing by industry professionals
Unlike AI, CHOICE Expert Ratings, recommendations and reviews are the result of extensive independent product testing by industry professionals.
Our testers scrutinise popular items across a range of key criteria, including performance in different areas, ease of use, privacy and, of course, safety.
We buy all the goods we review ourselves, are independent from product manufacturers and retailers and don't preferentially score a product just because it's from a certain brand.
For comprehensive information, independent advice and expert recommendations on a broad range of products and services, check out our reviews.
We're on your side
In more than 60 years of making a difference for Australian consumers, we've never taken ads or sponsorship.
Instead we're funded by members who value expert reviews and independent product testing.
With no self-interest behind our advice, you don't just buy smarter, you get the answers that you need.
You know without hesitation what's safe for you and your family.
And you'll never be alone when something goes wrong or a business treats you unfairly.
Learn more about CHOICE membership today
Stock images: Getty, unless otherwise stated.
