Notes get archived to the atproto wiki
Join the chat in the community dev Discord #indieskywg
Meetings are listed on the AT Community Fund Luma calendar until we can get Smoke Signal to replace it
Moderation is the product! As we talked about PDSes last session, moderation came up as a major concern when hosting data infrastructure for other users. This session we'll talk about moderation for ATmosphere:
- Moderating ATproto apps
- Bluesky's moderation and their Ozone labeler
- Community moderation, and communities currently running moderation (like Blacksky)
Please intro yourself if you haven't been before
- Andrew Chang, Product Manager Intern @ ROOST
- Rudy Fraser, Blacksky @rudyfraser.com
- Darius Kazemi, Applied Social Media Lab, W3C SWICG Trust & Safety
- Victoria, ATBrasil, QA Engineer and Community Moderator for some Brazilian Content Creators @vicwalker.bsky.social
- Sebastian Vogelsang, Creator of Flashes, Skeets and Bluescreen
- Clinton Bowen, Blacksky @torrho.com
- Hilke Ros, musician and coder @hilk.eu, trying to get more artists on ATProto
- Gautam Dey, Software developer. One of the creators of Tegola vector tile server.
- Zaria Jalan, Blacksky
- Sri, Software Engineer @sri.xyz
- bumblefudge.com, socialweb.coop, IPFS fdtn. (AFK)
- Mathew Lowry, Info architect for various clients in Brussels, not a developer, founded myhub.ai, launched first EC online community 2002 (hence grey hair)
- Ricardo J. Méndez, Numergent, @ricardo (attended the #001 but missed #002). Working on a small project but not public yet.
- Dave, building in stealth on ATprotocol since 2 months.
- Laurens Hof - writer about ATProto at fediversereport.com
- Matt Terenzio - Dev at Subtext by day - ATProto hacker by night (https://bsky.app/profile/librenews)
- David Fraser @david.frasergo.org, Inconceivable / Foodios (not public yet)
- Moderation and labeling could be shared effort, e.g. coop model for CARE (the relay).
- closed groups but avoiding net-split
- is there a global set of lowest common denominator obligations, or are the legal issues different in different geozones?
- What laws and regulations are applicable for your app?
- CSAM & law enforcement requests
- Copyright & Intellectual Property
- Are you using (and moderating) non-bluesky lexicons?
- What kinds of tools and service providers are available?
- Who is running your moderation?
- CARE
- Tooling
- Are there open source tools available to handle diverted moderated blobs (for law enforcement)?
-
Prior Art on federated CSAM issues.
-
Juliet: for the US, there has been recent legislation that changes data retention requirements related to CSAM detection - https://www.thorn.org/blog/the-report-act-explained/
-
Moderation has been part of the internet from the beginning. But different now with distributed social networks.
-
What is needed to run an app and what is needed to run in social spaces?
-
Comminties, and standards we want to build to.
- Standards are going to be different, based on regions.
- How you are storing data, what, type, and where?
- What do you have to do when the cops come around knocking?
- They have become more sophisticated in how software works, and what they can ask?
- i.e. asking for data, geolocated entries, how much to store? e.g. Google only stores 2 years of geolocated data.
- Sebastian talked to some Euro politicians about developer needs
- Robin: They are in active conv. with a legal team to get a solid brief, and what is needed. And are talking to politicians about what is needed, especially for local PDS's and moderations.
- Currently everything is under the bluesky lexicons.
- Robin: For some common stuff (like CSAM) a blob is a blob. The ability to do more complex more advance is very lexicon depended is still in development stage. (What is an easy way for developers to map their lexicons to map to common format. I.E. this lexicon describes a text or an image, etc...)
- There is a difference between building an app and host data for an app.
- Moderating video, is different then Moderating video clips.
-
How does Rudy and the Team think about running and Moderation of a community like Blacksky:
-
Rudy: 3 Pillers
- curations, setting community norms.
- remove worddal posts
- posts with the n-words with the hard r, does not get put into the database.
- %%missed what he said here%%
- community moderation
- extra-community v. intra-community
- extra-community e.g.: someone not in the community
- labeler + ozone is helpful here for reporting
- started with Rudy and one other mod.
- the first mod burned out. Now only brings in people who have experice in it.
- Something unique with bluesky is you can subscribe to what you want reported to you.
- With at-proto you have to deal with video, image, and text.
- text is harder, since you have to get the context. But Ozon is not condusive to doing that.
- Ozone is really building it for themselves, and not really for a community like blacksky. So, have been asking for more tooling for the community.
- Ozone has an appeal system, but that is, also, hard.
- Ozone does not have a way to communicate to the people that are effected by the community moderation
* the lack of channels for communication between moderators and people being moderated is a problem on the fediverse too https://fediverse-governance.github.io/#1.5-communication-between-moderator-and-user
- Platform Moderation
- data stored in different places
- PDS responsible for taking down
- Relay, information has a TTL - lower risk/concern
- Relay also does not store or cache media files, only atproto records
- exception: archival relays
- Client: Probably not as much responsibility
-
Juliet: Just throwing academic papers out because I am obsessed with them, the ABC framework that was designed for misinformation/disinformation is a good framework for thinking about T&S broadly https://www.annenbergpublicpolicycenter.org/wp-content/uploads/ABC_Framework_TWG_Francois_Sept_2019.pdf
-
Rudy: Where does the libility stand?
- Ted: There are different standards for different types of stuff. (may not matter in small scale, but has you gain scale, it does.)
- Ted: "if you do anything that touches blobs" then you likely have liability / obligations around it
- Trust and Safety is just a war of attrition.
-
Rudy: As of right now we don't have anything public, but internally we do have policies. When do you label. They have different types of moderations curation v.s. labeled.
- They combine labels on top of blueskys labellers.
- They setup a script to email all the moderators.
- How to deal with Mass stalking... people don't make a post, but make a gross account and then just mass follow people.
- Anyone to use Hide or Warn label. And these are very powerful, since once an account is labeled Hide or Warn it can not really unhide or unwarn.
- Rudy: Ozone has a setting of pulling in labels from other ozone instances. Defaults to Bluesky. And so if they label something as sexual it is blurred by default now which is v nice. Agree with everything Juliet is saying
-
Ted: This is why this is important. We need to know what is human problems, v.s. what is technology problems. How can we extend the technology to enable the people to more effiencently deal with the human problems.
-
Juliet (head of product for ROOST): intro; current projects:
- What are the core safty tools that need to do safty.
- Will be releasing an OS tool later this year for helping labeling; hopely will integrate will lots of other tools. It's already integrated with industry wide tools, including NCMEC (the National Center for Missing and Exploited Children in the US).
- There’s a whole rendering / display thing for reviewing reports, which is super gnarly
- Before launching this, there will be browser extension that will automatically blur / show images in a grid that it matches on a page.
- Trying to build all tools so that it is widely available to all platforms. So, initial it is not specific to at-proto.
-
Bryan on Ozone development - there's a combination of features, some targeted at larger teams, with rules about who can / can't override things, but also hoping to serve the rest of the community. It may be the case that when it gets to the point where a different storage backend is required to scale, that they need to fork and have a separate internal/external version. Ozone also has a backend and then a UI; most of the feedback they get is on the UI.
- It would be good to add a lot more functinoality related to moderation into the app instead of having to do it through Ozone.
- They tend to prioritise legal requirements that come up, which hopefully helps other teams too.
- Have stuck to doing post-moderation so that content coming from someone else's PDS is treated the same as content coming from Bluesky PDS.
- It would be nice if external PDS could delegate some of the pre-emptive detection for extreme content like CSAM, if they so choose, so that they don't have to deal with it after it's already visible. That isn't available right now.
- Have started building a bit of a web interface to the PDS; it would be a natural place to add features to take accounts on the PDS down; it's intended that people use Ozone but if they don't needs to be done through a command-line tool at the moment.
-
Juliet: Recommend checking out https://www.tspa.org/curriculum/ts-curriculum/
- They’ve written a ton of stuff down to hopefully help people get started and skip the painful lessons of safety/moderation
- What Bryan is describing is what T&S folks (or at least I) call post-moderation. There’s also pre-moderation, so pre-moderating something before it’s shown to anyone
- Some platforms do like a mix, where it’s live before moderation, but it gets reviewed once it is seen by x people. Rinse and repeat as something gets more popular
-
Rudy: if you ran a pds you can wire it up to ozone instances right?
- Bryan: I don't think that is quite there yet. There's a change account status API point; the intent is that you'd be able to send takedowns either to the relay or to the PDS, but can't remember if it works yet.
-
Bryan and Sabastian: talked about post v.s pre moderation
-
Juliet: For folks who have been doxxed or face online harassment, there’s a nonprofit (danish american) that help act as a Trusted Flagger (aka prioritized reporter) on platforms since attacks are often sharded across different platforms https://www.pirth.org/
- Talk to the lexicon community about additional metadata that would help to identify categories of data for moderation purposes
- Juliet: I’d love to see a decentralized version of this game that explains how moderation works https://moderatormayhem.engine.is/