The social media platform Bluesky recently had an incident where a user created an account with a racial slur as the handle. The Bluesky team quickly removed the account but realized they should have had automated filters in place to prevent such issues. They are now implementing a two-step automated filtering and flagging system for user handles while still involving human moderators. The team acknowledges they were too slow to communicate with the community about the incident and are working to improve their Trust and Safety team and communication processes going forward. They are committed to learning from this mistake and building a safer and more resilient social media platform over time.
Previous post about this topic https://beehaw.org/post/2152596
Bluesky allowed people to include the n-word in their usernames | Engadget
Bluesky, a decentralized social network, allowed users to register usernames containing the n-word. When reports surfaced about a user with the racial slur in their name, Bluesky took 40 minutes to remove the account but did not publicly apologize. A LinkedIn post criticized Bluesky for failing to filter offensive terms from the start and for not addressing its anti-blackness problem. Bluesky later claimed it had invested in moderation systems but the oversight highlighted ongoing issues considering Twitter co-founder Jack Dorsey backs the startup. The fact that Bluesky allowed such an obvious racial slur shows it was unprepared to moderate a social network effectively.
Failing to attempt to design and impliment an important feature at all is not the same as a bug. Unless I’m missing something they aren’t saying “we did have systems in place to prevent people creating accounts with intentionally offensive usernames but we oopsed so it didn’t work as intended until we fixed it.” They’re saying “it either didn’t even occur to us our software needed that or we decided we just don’t care so we didn’t even try to do it until people pointed out that we were missing this important thing at which point we started working on it.”
So, either they somehow just missed that this is something they need (which they really shouldn’t have and suggests they aren’t thinking even slightly about user conduct on their platform) or they did and decided they wanted to see if they could get away with just not doing it.
I understand it’s easy to get lost in the core functionality of making the thing go but you can’t lose sight of the actual intended outcome like this.
I suspect it may be a bit more along how you’re describing here – we expect some user experience patterns to already be in place, if not considered, like not being able to select inappropriate handles. Former Twitter folks should know ‘better.’ From the outside looking in, it tracks.
I wonder if the Bluesky team, right now at least, is more engineer / dev heavy, and they have not brought on UX folks to help drive a product design that considers patterns we’d be used to experiencing. They may be operating pretty lean.
An idea, at least.
I think that’s quite likely yes. But that in and off itself indicates that management consider stuff like UX to be non-essential expertise that sits outside of what is required for a functioning lean operation.
Not entirely true, there are many other factors such as timelines to deal with. I am a professional developer and I have had projects where the timings were tough (competition, sometimes just contract/SOW delivery date changes, etc) and UX was specifically disengaged or delayed.
Now is the ideal time to strike, fediverse and IG threads/lemmy/mastodon/etc are all still in flux when it comes to community favorites. They are all motivated to get out to market first.
The thought behind this is to get the foot in the door functionality wise and revamp UX overtime based on user feedback once established in addition to internal evolution of UI. It’s not like they said “yeah fuck it let’s let ‘em name themselves anything”, and more likely prioritized issues in the backlog and being a startup, have more flexibility to leave UX for later or losing the finer details of community safety to different phases or whatever.
It’s truly difficult to point to the cause from the outside looking in, yet many here are happy to take this whole situation as evidence of either malice or incompetence. These days when you release a product it’s not done.
These days the following are common: Day-one patches, frequent weekly/monthly/etc. update cadences, pushing to dev/dit/qa/prod rapidly, always keeping production in line with or not far behind develop (aka, merging to dev, run CI/CD, waterfalling thru each environment all the way to prod).
I submit that this whole situation was a failure of management as you said but not the disaster people are trying to make it out to be. These things happen when the dev team is forced to move quickly, it says nothing about the company’s values or what it really cares about. They clearly care, it’s just that they see getting to market as the penultimate hurdle that all others are secondary to. I don’t advocate for this approach as I prefer the show not tell approach but I understand the thought process.
Bluesky is not even released yet, it’s still in early beta. How can anyone speak to their priorities and what is or isn’t essential to them when the product is still in active development with no release date in sight. Their sign up is a waitlist and the hosting provider (first question they ask) has options for dev and staging servers.
I would say it’s likely they are very lean. From what I’ve heard it isn’t more than a few people closely working with Jack Dorsey full-time right now. Here’s a blog post from last year with some hints as to the size of the core team.
While they definitely know better, it’s a closed beta and most of the users have already been vetted prior to invitation. The fact that someone made a bad name means they were testing viability, which is what a closed beta is for. A team of even forty or fifty people working on a fresh project have plenty of other problems and issues to address, even if username filtering is an important one.
We don’t know they didn’t design and implement it. Happens all the time where you imolement a feature, it works, there’s a regression and you have no clue. 40 min to resolve means it’s there, no way you’re building that completely, testing it, oushing it in that tineframd.
I coukd be wrong.
It says in the post:
“realized they should have had automated filters in place to prevent such issues. They are now implementing a two-step automated filtering and flagging system for user handles while still involving human moderators.”
They wouldn’t need to implement a system they already implemented but wasn’t working properly. They’d just be fixing it.
I misinterpreted that part or missed it. With you now. Thank you!