Need to know
- Airbnb has bought an algorithm to score users’ ‘trustworthiness’ based on publicly available data
- Some users in Australia have been banned from the platform, but the company is vague on how it applies the algorithm here
- Experts are concerned by the growing use of automated decision-making in Australia
If you've ever been in the market for a holiday rental property in the past few years, chances are you've been on Airbnb.
The US-based platform has become a global behemoth in the short-term rental market, with 5.6 million properties listed on its website around the world in 2021 alone. More than 800 million people worldwide have stayed at an Airbnb property.
But while the company has become synonymous with holidaying, some users in Australia are finding themselves blocked and banned from the platform, and are asking why.
Airbnb acquires 'trustworthiness' algorithm
In 2017, Airbnb acquired a tech startup called Trooly which specialised in background checks and had earlier patented an algorithm that gathered publicly and privately available data on users to give them a 'trustworthiness' score.
Kate Bower, consumer data advocate at CHOICE, says the data the algorithm uses to assess your 'trustworthiness' includes social media accounts of yourself and your friends, what you do for a living, your education level and any other online data it can find.
Assessing personality traits
The patented algorithm is claimed to assess people's personality traits, such as narcissism or conscientiousness, along with behavioural traits, such as use of drugs or alcohol or involvement in civil litigation and other behaviour, and combine them to create a holistic score that judges a person's trustworthiness.
Airbnb has updated the patent, which it owns directly, several times since 2017. This suggests it's using and developing the algorithm, not just letting it languish on some back shelf.
Airbnb's privacy policy includes a broad statement that indicates the algorithm may be in use in Australia. It states, "we may conduct profiling … [and that] automated processes, which analyse your account and activities on the Airbnb platform as well as information in relation to activities on and off the Airbnb platform that can be associated with you, could restrict or suspend access to the Airbnb Platform".
Customers report being blocked for no reason
Renae Macheda describes herself and her husband as "clean, boring people". She works in real estate and was baffled when she tried to book accommodation in Sydney in October – only to find herself banned from Airbnb and her account closed.
The company gave her no reason for the ban. When she queried the decision, she got a brief email that was no more illuminating.
It read: "After reviewing all the information available to us, we've determined that your account will be removed from the Airbnb platform. Removal means that your account will no longer be accessible, and you won't be able to create another one. We want to assure you that we reviewed your case thoroughly before reaching this conclusion. As such, we won't be able to offer you additional support on this matter at this time."
'It's not really good enough'
Renae tells CHOICE she's never had a dispute with an Airbnb host and has had nothing but good reviews from her many holidays. She says she's "devastated" by the decision and feels let down by the company.
"I think they should give you an explanation," she says. "To give nothing at all and no options to try and remedy whatever it is, it's not really good enough."
Airbnb removed Renae Macheda from the platform without explanation.
Technology experts say: 'Be more consumer-friendly'
Ellen Broad from the Australian National University (ANU) College of Engineering & Computer Science says companies like Airbnb, which operate similarly to public utilities, ought to be bound by a similar set of responsibilities.
"They have a responsibility to give people meaningful avenues to understand why a ban occurs and meaningful avenues to appeal then, but they're not doing that," she tells CHOICE.
They have a responsibility to give people meaningful avenues to understand why a ban occurs
Ellen Broad, College of Engineering and Computer Science, ANU
Julian Thomas, from RMIT and director the ARC Centre of Excellence for Automated Decision-Making and Society (ADM+S), also expresses concerns about the algorithm's potential use in Australia.
"We need to be looking at how we can regulate these systems better to ensure that they are much more consumer-friendly, that they do respect people's rights, that they are more responsible," he says.
Sex workers banned
Rick Andrews (not his real name) is a Melbourne-based erotic masseur who found himself banned from Airbnb last year when he tried to book a holiday in Sydney.
He says he had nothing but good reviews from the dozens of local and international holidays he's booked through the platform.
I was outraged, this is unbelievable, what have I done? Murdered someone?
Rick Andrews (not his real name), Melbourne-based erotic masseur
Airbnb sent him an email saying his account had been flagged during a "standard security review".
The email read: "It turned out that your account was linked to activity that goes against our Terms of Service, specifically it was linked to online ads for adult services, which can include escort activity and commercial pornography."
Rick says, "I was outraged, this is unbelievable, what have I done? Murdered someone?"
Jane (not her real name), a Melbourne-based sex worker, says her account was blocked in 2018. She believes it was due to it being linked to online ads for sex work, and says "there was no explanation given to me at all".
Ban could be discriminatory
Matthew Roberts from Sex Work Law Reform Victoria says that banning someone from a platform based on their occupation is discriminatory and should be protected against.
"I think we should all be aware of what Airbnb is doing before we sign up to Airbnb, that's not being disclosed or made transparent," he tells CHOICE.
Gala Vanting of Scarlet Alliance, the national representative body for sex workers, says they've also seen a large number of cases of sex workers being banned from Airbnb.
"For many sex workers removed from Airbnb, there's no process, no violation of their terms, no story to explain their loss of access – and no appeals process to get an explanation or reclaim an account," says Vanting. "Airbnb's silence around these cases is deafening."
Airbnb vague on algorithm use in Australia
We sent a list of detailed questions to Airbnb Australia, hoping to find out more about the way its algorithm is being used in this country.
But the company's response was extremely vague. It said: "The safety, security and privacy of our community is one of our top priorities. Our platform security and safety measures are designed to help ensure stays are safe and positive experiences for Hosts, guests and the wider community – while also protecting users' information, including their personal information."
'Lack of transparency'
Bower says that this response isn't good enough and that Airbnb has a duty to inform Australian customers how it's gathering and using their data.
"There is a worrying lack of transparency of potentially harmful and invasive practices," she says. "Airbnb claims it offers users an opportunity to have automated decisions reviewed but in reality people are left in the dark about why they have been removed from the platform."
Some users are locked out from Airbnb by opaque automated decision-making.
Use of algorithms on the rise
Kimberlee Weatherall is professor of law at the University of Sydney, and a chief investigator with ARC Centre of Excellence for ADM+S. She says there are several concerning things about Airbnb's algorithm and its use of data unrelated to people's Airbnb rental history to judge how they'll behave in a property.
"Using what they did at university two years ago to decide whether they get holiday accommodation tomorrow, that crossing of contexts, I think is a real problem," she tells CHOICE. "These algorithms either punish people for the behaviour of others, or punish people for their past behaviour."
These algorithms either punish people for the behaviour of others, or punish people for their past behaviour
Kimberlee Weatherall, ARC Centre of Excellence for Automated Decision-Making and Society
She likens this "crossing of contexts" in data use to China's controversial social credit scoring system, and says we should be concerned.
"This idea that you get surveilled across all different kinds of contexts and bad behaviour on one metric can lead to punishment is concerning," she says.
"In the China example, you didn't pay your fines, so you're not allowed to get on a train. If we move to a world where we say that it is OK for Airbnb to surveil you across multiple contexts then people will have to start censoring themselves online and what they post."
Dangers of 'social scoring'
Bower agrees with Weatherall, saying the "algorithm amounts to a form of social scoring reminiscent of a Black Mirror episode" (Black Mirror is a British TV sci-fi series that focuses on the dystopian use of technology, often in the near future). Social scoring, sometimes called 'social credit', is when an automated system assesses a person's trustworthiness or likely future behaviour.
Just as credit reporting aims to judge a person's ability to repay a debt from their past financial behaviour, so social scoring algorithms attempt to predict how likely a person is to behave in a certain way based on their past behaviour, as gleaned from their personal data.
Social scoring can reproduce societal biases and lead to real-life discrimination. "Most Australians would be shocked that profit-driven businesses like Airbnb have appointed themselves moral arbiters of their behaviour," says Bower.
Regulation needed
Automated decision-making is a computerised process that can either replace or assist human decision-making, often using advanced data analytics and machine learning.
Most Australians would be shocked that profit-driven businesses like Airbnb have appointed themselves moral arbiters of their behaviour
Kate Bower, CHOICE consumer data advocate
Its use is growing around the world, including in Australia. Bower says, "It is already widely used in credit reporting, insurance and financial services and we are likely to see it increasing in other sectors including housing, healthcare, retail and many others. It has great potential to improve services for consumers but it is also potentially harmful."
The EU model
Thomas from the ARC Centre of Excellence for ADM+S says Australia needs to look at regulating the way companies use cross-contextual data for automated decision-making. He says Australia could follow the line of the European Union (EU), which is currently addressing concerns around automated decision-making and social scoring.
"We [Australia] are a small market, so it makes sense for us to follow the lead of the EU on this," he says. "All global companies will have to interact with the European market in some way and their policies are often global. If we think the EU framework does provide strong consumer safeguards, I think we should work with those."
CHOICE pushing for better protections
Our submission to the Privacy Act review supports the adoption of the EU framework, which proposes an outright ban on social scoring algorithms, along with regulation of other uses of automated decision-making and artificial intelligence that pose a high risk to consumers.
According to Bower, Airbnb's potential use of its algorithm in an unregulated space is highly worrying.
"CHOICE is concerned that businesses are implementing automated decision-making widely without informing consumers of the risks, offering avenues to opt out or opportunity to review decisions," she says.
"Automated decision-making has the potential to make inaccurate or biased decisions and lead to discrimination and unfair practices."
Do you think it's unfair that Airbnb can use your personal data in automated decision-making to restrict you from the platform? Answer our poll.
Stock images: Getty, unless otherwise stated.
