COllaborative DEcision MAking (codema.in)

Our position on probabilistic automation ("AI")

fugata Wed 18 Nov 2026 8:16AMPublicSeen by 4

Here's a post I've drafted. I'm posting it as a new thread so others do not need to read the very convoluted discussion in the previous thread. It can probably be refined a bit, but the basics are solid.


It is long overdue for FSCI to release a statement on the current storm in the software world - that of probabilistic automation, a.k.a. "AI", especially the generative kind.

We preface this by reminding everyone about the original values of the free software movement - upholding ethics, strengthening community, and preventing unjust hierarchies (a.k.a. monopolies).

Terminology

"AI" is a misleading marketing term which obscures multiple technologies into an umbrella term, and makes it difficult to discuss their merits and demerits.

We recommend using "probabilistic automation" instead (see How to talk about "AI" without adding to the anthropomorphization), or specific terms such as LLMs, latent diffusion, etc.

Unethical production

This is a rude technology, and it deserves a rude response.

— Anthony Moser, I Am An AI Hater

The numerous issues with prominent contemporary examples of probabilistic automation begin with its creation -

  1. The destruction of the independent web.

The constant onslaught of scraping bots has driven many websites to use privacy-invading, proprietary and centralized services such as Cloudflare. Users of Tor have been particularly impacted. Other services have login-walled their content.

Costs of running a website have been driven up. Smaller sites have vanished entirely, their admins burned out mentally if not also fiscally.

  1. The violation of copyright and consent.

Websites scraped without consent. User-submitted content sold for LLM training without (opt-in) user consent (and in many cases without even a way to opt-out).

(In many ways, training of LLMs was enabled by centralized services. The creation of Copilot would have been significantly more difficult without the existence and mass adoption of a centralized service like GitHub.)

  1. The violation of the free software socio-legal contract.

Free software does not mean a "free-for-all"/anything goes. Permissive licenses require attribution of derivative works. Copyleft licenses require share-alike licensing of derivative works.

We consider LLM weights to very obviously be a derivative of the training data. The training data must satisfy attribution, share-alike, and Affero clauses, as applicable.

(Some foolishly celebrate this as a supposed weakening of copyright. However, we point to the fact that all mainstream probabilistic automation is proprietary software with proprietary weights. As long as big companies continue to enjoy copyright protections, it's highly unfair to deny the same protections to smaller players, and utterly naive to assume that copyright is going anywhere simply because some big players temporarily and selectively decided to forget it exists.)

  1. The destruction of physical books

  2. The use of slave labor for training.

  3. The ecocide - the massive consumption of energy, hardware, and land, incomparable in scale to any other technology.

We are not currently aware of any "ethical" probabilistic automation, i.e. which does not run afoul of the above criteria in one way or another. Models trained only on public domain data are nowhere near as useful as mainstream models. The technology, at its core, relies on extractive exploitation - no "ethical" PA can escape that.

Unethical results

So that's before we even start using probabilistic automation. What are the social effects when we do?

  1. Privacy - the best-performing models are centralized services
  2. The weakening of communities
  3. The deskilling/cognitive surrender of users and programmers, and increasing dependency on PA companies
  4. Burnout of free software maintainers
  5. Mass unemployment, and the weakening of labor power - in other words, entrenching the existing hierarchies of the world
  6. The (easiest yet) creation of disinformation and propaganda, used to weaken our democracies
  7. The (easiest yet) creation of deepfakes, primarily to abuse women
  8. Misleading users, resulting in accidents and death
  9. Harassment of innocent citizens when used by the police

Another serious consideration is sustainability - LLMs cannot be trained on their own output.

LLMs are trained on community-owned resources like chat, forums, wikis, and mailing lists. If people use LLM chatbots instead of these community-owned resources, the latter will grow stagnant, eventually rendering LLMs themselves less useful.

Likewise, PA chatbots have been trained on web content, but with the deluge of slop content, identifying human-made content has suddenly become very important to all the PA companies. (see e.g. https://c2pa.org/)

Thus, LLMs are inherently unsustainable.

But the benefits!

The claimed benefits are also dubious.

Probabilistic automation (as the name implies) is inherently lacking in intelligence and has no understanding of its output. Nothing prevents the output from being plausible-seeming garbage.

On one hand, adopters claim boosts to productivity. On the other hand, adopters try to justify their actions by claiming that they rigorously examine and test the LLM output...but it has (famously) always been faster to write software than it has been to study it. Perhaps unsurprisingly, adopting companies have reported only marginal improvements in productivity.

However, the claimed benefits are besides the point. No benefits can justify the violations of ethics. In an unethical world that says "the ends justify the means", we say, "there are no ends, only means."

What would an ethical PA look like?

  1. The model itself has to be libre software
  2. The training data used must be declared
  3. The training data must be libre data
  4. The training process must not use slave labor
  5. The weights must be released as libre data
  6. The weights must be treated as a derivative work of the training data
  7. The model must run locally and offline, or be self-hostable on consumer hardware

N.B. We haven't specified any environmental considerations. It is not clear if there can exist a model that does not consume vast amounts of land, water, and energy.

Conclusion

We consider generative PA to be an attack on the free software movement and an insult to all our values. Using it for any purpose is a distasteful act.

Our recommendation is to oppose unethical PA at every step, as with opposing proprietary software and centralized services. Improving human skills and knowledge, and improving community-owned resources (chat, forums, etc) is vastly preferable to outsourcing your thinking. Individuals should boycott it, along with free software projects that use it. Those forced to use it at work should form unions to oppose its use.

We laud projects like Ethical FOSS, Codeberg, and many others for having taken a stand against probabilistic automation. We also express our disappointment with the "open source" and "FOSS" developers and projects (Debian, KDE, Firefox, ...) who have forgotten their roots and ignored ethics in favor of short-term convenience and overhyped claims of productivity.

Badri SunderarajanWed 18 Nov 2026 8:55AM

N.B. We haven't specified any environmental considerations. It is not clear if there can exist a model that does not consume vast amounts of land, water, and energy.

I don't think this is a good reason to avoid specifying environmental considerations. One could similarly argue that we should ignore user freedom considerations when deciding whether to use Microsoft Windows, because it is not clear whether there can exist a version of Microsoft Windows that respects user freedoms. We are setting the standards: it is up to the technology to be able to meet those standards—or not, in which case it can be skipped in favour of a different technology that does.

fugataWed 18 Nov 2026 9:22AM

@badrihippo Sure, I just elided it to get the draft out quickly :) What considerations or metrics would you suggest?