the point here being small service providers are _also_ able to get into this system, but I think there's an expectation of success.
The bar for trusted publishers is very high though. I don't think it's high "on purpose" to leave out certain platforms for being too small, but in effect if (for example) your provider doesn't have a security contact point then that's going to rule them out here.
I don't think this is true. From my experience, developers are the people who care about this topic the most. Just look at HN, almost everyone (who speaks out) is against.
I'm very in favor of this work, but don't generally comment because these comment threads are full of folks strawmanning the work being done, and in my experience those folks aren't really interested in productive discussion.
I'd be wary of trying to estimate the split of opinions based on who comments.
So what? Ultimately the trust model is supposed to be there to validate that this latest blob of code is coming from the same source as the last blob of code for the same project, authenticated with the same public key. And that's all it is telling you, and all that it can tell you, whether the attestation is generated during a Github workflow or by a program running on my personal computer that you can't see. It is not telling you that this latest blob of code is safe and free from problems. It's just telling you that it comes from the same source.
The average person using GPG isn’t actually providing me much evidence in that regard, given the number of ways to have technically used a GPG key to sign an artifact that aren’t secure.
That's easy to handle: if the signature isn't secure, it doesn't count as an attestation. I have no problem with putting the onus on the person doing the signing to do it right. But at present doing it right on your own is not even an option. Only going through one of the tech giants is.
They can't. But they can't tell for sure if large tech corporations are doing that either. Data breaches are a thing. Auditing Github workflows doesn't prevent them.
1. "ClosedAI" is not gonna win people over to your side, IMHO. Apple doesn't plant trees, Microsoft makes big hardware, and Python is not satanic.
2. I'm not entirely sure what the link is supposed to prove -- would appreciate a word of clarification if you find the time. It almost feels like a dogwhistle of some kind, but A) I doubt anyone on HN is anywhere near that problematic, and B) there's no clear pattern in those companies other than "interested in ID tech", and none of the usual dogwhistle targets made the list. Also, y'know, OpenAI isn't on it...?
I guess my fundamental confusion isn't about the list and more about what "we know". Are you implying that they are against pure, unmanaged PGP for this system because... I mean, why? Because they're playing the long game to promote a verification/attestation/identification (?) technology that you think offers less stringent privacy guarantees, and thus to sell more ads in 2036?
If so, doesn't that seem like a kinda roundabout way to raise Display Ads revenue, otherwise known as "that one dying industry"?
It's anti lots of things, but I don't think it's anti-Pythonic. Python is about pragmatism & empathy if it's about anything -- bitcoin and such don't factor into it. See: Astral/OpenAI, Anaconda, most of the PSF board historically being employees of big corporate users of python, NVidia's relation to the ecosystem in general, etc.
As a passionate anarchosyndicalist, I, for one, am thrilled about it. Decentralization is a tool in our toolbox for identifying and dismantling unjust hierarchies, but it is not a panacea, and can be quite damaging when taken to the extreme. See: cryptocurrency speedrunning 200y of bank regulations in 20y.
To put my trust for Python in more concrete terms: they support GitLab. As a passionate GitLab shill, I really couldn't be more impressed! It continues to shock me how infrequently the second most popular forge is acknowledged, much less supported. This shows they're not just, IDK, bowing to Microsoft's demands.
> This shows they're not just, IDK, bowing to Microsoft's demands.
Not only don't we, but I must emphasize that Microsoft did not pay for anything involving the implementation of either Trusted Publishing or attestations on PyPI. I wrote a personal blog post explaining some of my own rationale for why we prioritized GitHub and GitLab for PyPI's initial support targets, and it shouldn't be surprising to anybody who knows me that the reasons are pragmatic, not dogmatic[1].
Is "a public key that I personally upload to my PyPI account, using the credentials you already have that allow me to use that account" on the list of targets that will ever be supported?
If not--i.e., if only recognized "forges" are ever going to be supported--why not?
All absolutely reasonable. At the same time though there are a lot of roads leading to centralisation, and as they all converge it gets just a bit .. problematic.
Absolutely not claiming its not a pragmatic step, but equal-footing alternatives would be useful.
I'm no tin-foil-hat - but when a website account requires government id (like in the uk), and creating software requires signing by the website, you link all the things - is that a good outcome?
This is true, and it’s why I’m hopeful that some of the newer forges can make meaningful default posture improvements that make migration away from centralized services the dispassionate choice to a user who cares about availability and security.
This is great! I want sigstore-based or at least sigstore-style "keyless" short lived signatures to become more and more popular, so that support gets better everywhere. In particular, if having secure OIDC handling in a CI/CD pipeline becomes something that keeps people on GHA or GitLab then I'd hope third-party hosted CI/CD services begin to take that gap seriously and work on meeting those requirements.
I'm bought in on keyless signing, and I want there to be many more providers, so I like to see more adoption which builds more momentum.
>I'd hope third-party hosted CI/CD services begin to take that gap seriously and work on meeting those requirements.
How is a third party service, especially a self-hosted one, expected to meet those requirements? They seem to be lengthy, onerous, and also require a detailed vetting by the pypi team that is unlikely to scale beyond a few large hosting providers.
(What's more, this does relatively little to actually signal trust to someone downloading them. All it does is provide a signature that one of the big providers has built the package from a given source (but nothing about the trustworthiness of that source). This is a problem much better solved by reproducable builds and distributed verification of those builds)
I think OIDC tokens from workflows is basically table stakes at this point, since the alternative is a static secret, which is basically guaranteed to leak eventually.
All that's left is for PyPI to accept these tokens in their set of trusted publishers.
Please see the other comments in this thread (as well as on PyPI's issue tracker) for some important context here. It's not enough to throw JWTs over the wall, PyPI also needs to know that it can trust any given service to operationalize the ugly and risk-prone parts of managing PKI.
(I haven't looked at Tangled at all. If Tangled has a secret management story for its PKI that's stronger than a private key on disk and can demonstrate that a non-trivial number of PyPI users would benefit from federation, then it's a strong candidate.)
Sorry I didn't intend to insinuate that pypi should necessarily accept any old tokens just because they're there. Just that increasingly OIDC is more and more an option, which is generally a good thing.
PKI definitely is still hard, but if it can be done well by a smallish number of trusted parties to the benefit of their users, this can have huge ecosystem benefits. Let's hope we live in that timeline.
Very cool to see PyPI put more emphasis towards exposing provenance info. It would be even cooler if trusted publishing would finally support reusable workflows (https://github.com/pypi/warehouse/issues/11096).
It would be even nicer if there were a way to have an attestation without publishing with a workflow controlled by one of the tech giants. There's no way for me, someone who has had a PyPI account for many years and has used it for all that time to upload packages, to attach my own public key to my PyPI account and then sign uploaded distributions with my private key so the signature can be verified and count as an attestation.
I think the problem is this type of attestation is hard to weigh as a trust signal, vs the cloud CI version.
Inherently, if someone steals your key, or gains access to a box that can use that key for signing, they are indistinguishable from you. Hand PGP signing also lacks time-bound signing, any ratchet mechanism, CT log, consistent key hierarchy, or standard to bind signer name and project identity.
> this type of attestation is hard to weigh as a trust signal, vs the cloud CI version
In the sense that I'm not a large corporation and nobody else is auditing my personal computer, yes, that's true. Note, though, that large corporations are also subject to nefarious actors misusing their infrastructure. What if a rogue employee hacks their internal system? Or Russia or China manages to get a backdoor into it?
In any case, all this doesn't make the method I described impossible. It just means that attestations produced by an individual workflow need to be clearly labeled to distinguish them from attestations produced using a tech forge's workflow, so people will understand what, specifically, is being attested to. Then people could weigh each trust signal as they choose.
> Hand PGP signing
To be clear, I wasn't talking specifically about this method of signing with a personal key. I agree there are lots of issues with PGP as an overall system. I was only talking about the general idea of having some way for individuals to independently generate an attestation without having to go through a tech forge's workflow.
This is about detecting formerly benign projects that have had their pypi entries hacked, but not their private keys. It does not detect malicious projects.
Am I correct in my understanding that the only trusted publishers are for-profit US companies? And these all use proprietary implementations which cannot easily be replicated elsewhere?
Like, if I want to host on Codeberg, or Sourcehut, what is the process needed for those platforms to become a trusted publisher?
As it happens, I build my releases on my own hardware using a pypa/manylinux release. Can I provide attestation, or am I required to use a third-party service to get that "x Without provenance" removed from my entry?
Support for Codeberg is being discussed here[1]. I think there's broad consensus among PyPI's maintainers and admins that more publisher providers are desired, but we also need them to demonstrate strong key management and rotation controls due to the degree of trust and access that becoming a publisher provider grants them (effectively, they become the PyPI credential broker for the projects that use their service).
(If you pan through that thread, you'll note that one of open issues with Codeberg is that their underlying server wants to store OIDC private keys on disk at the moment. This is okay for small deployments of personal services, but not for something that PyPI needs to federate with and trust.)
So I, as someone who does my builds locally, can never have attested builds?
Will PyPI be presenting warnings or statements of caution which might cause users to be concerned about using my distributions? Or imply that my build practices are somehow inferior to depending on a third-party build system?
The current formats (publish and SLSA provenance) are for attesting publishing and build steps, respectively, but there's nothing stopping PyPI from adding a different, new attestation statement for local builds (something that conveys "I did this locally, and I was this principal (email address) when I did it").
Attestations are code-signing. PyPI once supported PGP signatures on uploads, but as since disabled it because (1) only a tiny minority of users ever signed, and (2) many of the signatures and corresponding keys were either functionally insecure or unusable.
Both of these are latent problems when trying to use PGP in packaging ecosystems; the subset of ecosystems that successfully adopt PGP tend to have very different topologies (namely, small rings of trusted...publishers) rather than an "anyone can publish" architecture like PyPI has.
There's a discussion thread for email attestations here[1]. Those would support local builds.
But to take a step back: if you want attested builds because you think attestation is itself a gold star, then you're working from the wrong priors. Attestations do not imply that a package is secure any more than a hash does; what they are is a cryptographic signature over a set of claims, which allows for authenticated auditing of a project's state over time.
I think PyPI's documentation does a good job of conveying this nuance[2], and that much of the opprobrium of attestations comes from people either not understanding that nuance or imputing conspiracies into it.
> what they are is a cryptographic signature over a set of claims, which allows for authenticated auditing of a project's state over time.
Which is great--but does not in any way require that the attestation can only be produced by working through one of a small set of recognized "forges". You could do this just as easily by, for example, letting me upload to my PyPI account a public key (the same as forges like Github do to allow SSH access), and then letting me sign builds that I upload using the corresponding private key. But that kind of workflow is not being supported. Why not?
> Which is great--but does not in any way require that the attestation can only be produced by working through one of a small set of recognized "forges"
That's true, there's some context elsewhere in these threads about how PyPI can enable email attestations. The short version is that PyPI wanted to make this work by default, which was trivial with Trusted Publishing (since it already provides a strong source of verifiable machine identity).
"Why not a public key?" is a good question, and the very short answer is that greenfield signing designs should not assume that all users are able to manage the secrecy of long-lived key materials, much less force them to make parameter decisions when (re)generating materials. This is the same mistake that the Web PKI made for decades, and has spent the last 15 years slowly untangling. The ideal setup is one where users demonstrate control over an identity, and then the signing scheme binds that identity to a short-lived signing key. That's how email attestations work.
From what I can gather from the github issue that was linked to, those would still require the email identity to be vouched for by a provider. (Though if whoever already is hosting my email could be a "provider", that would at least greatly expand the list of providers from the current small number of forges.) There doesn't seem to be any way for me to, for example, link a public key I generate myself to an email address that PyPI already has on file for me.
Have you considered that perhaps you are working from the wrong priors?
I believe your prior is that you expect people who read the PyPI page to understand what attestation means. I believe your prior is they will see a notice about "not from a trusted publisher" and not treat it like demerit or warning. I believe your prior is that seeing this message will get people to do more due diligence than they currently do, and that the due diligence will be relevant and appropriate.
Based on my experience with my users, I don't have the same confidence.
I don't understand what "authenticated auditing" means, so why should they?
What project state do people think is important to audit? My code repo is private, hosted on Sourcehut. My build system is private, done locally. The only thing I release is the pre-built binary package. There's very little state to examine, and I can't figure out why anyone cares.
I also have customers who buy a source code license. They have never asked for an audit of the project's state over time.
I read the "Security model and considerations" document. I found it opaque, no doubt because the entire topic is foreign to me, but it doesn't do a good job of explaining. Take the second paragraph:
"The most basic type of attestation (the type enabled by default in pypa/gh-action-pypi-publish) asserts that the project was published by an authorized publisher such as a particular CI provider in a particular code repository.".
I know pypa/gh-action-pypi-publish is some sort of GitHub thing. I don't use GH or third-party CI so that clarification does not help.
It doesn't describe what "authorized publisher" means, other than a generic example of "particular CI provider in a particular code repository". That phrase exists only once in the document. Is it different from a "Trusted Publisher"?
Why are the terms "authorized" and "Trusted" used, instead of something like "affiliated"? If someone is not a trusted publisher or an authorized publisher it's is too easily misinterpreted as "untrusted publisher" or "unauthorized publisher", which seems scary. If someone is not an affiliated publisher, "unaffiliated publisher" sounds less scary. (I'm not saying "affiliated" is the right word, only an example of a roughly appropriate word where the negation of it is not so scary.)
Then later, "It protects against modification of a project after it was built".
I think that's supposed to be "modification of a release after it was built"?
My use of "project" matches the one at https://docs.pypi.org/using-pypi/pypi-basics/ "A project is a collection of releases and files, and information about them. ... A "release" on PyPI is a specific version of a project." I want to modify the project with updated metadata as things change, and to make new releases.
Oh, and I can't find a description of what "build" means in that document. It's used as if everyone expects to know what it means, but a few years ago I learned that modern Python packaging uses "build" different than I do.
When I compile my Python/C extension I "build" it. When I make a wheel I "package" it.
I believe that modern packaging calls the whole thing "build"? Or just the latter step?
Incidentally, "python -m build --sdist" on my project does not work as a replacement for "python setup.py sdist" because my setup.py includes a code generation step to get from a clean repo to the Python/C extension files included in the distribution. I haven't figured out the modern way to handle that. I think it's because of the differences in philosophy of what "build" means, and what actions are allowed during different stages of the distribution process, which is why I think "build" needs a specific definition, just like "project" and "release" have specific definitions.
I mean, my actual prior here is that nobody should care or even think about this stuff. Authenticity and minimally-scoped credentials should be ambient properties of a system, not something that users even need to read docs for, much less enable. But if we're engaging with the premise that users are looking at this stuff at all, then I think it's facially reasonable to expect them to read the docs.
For context: attestations aren't something you even really enable. They just happen, by default, if you use Trusted Publishing from a forge like GitHub or GitLab (and hopefully, more in the future).
FWIW, your feedback on the docs is very helpful. It's genuinely useful to hear that the phrasing in it is imprecise, and I'll try and improve some of it this week. The main confusion that I think you're having (which is IMO reasonable) is that the doc was written to be theoretically splittable from the "Trusted Publishing" concept, i.e. so that "publishers" (principals who can publish) who aren't tied to a Trusted Publisher can attest to their published files.
> nobody should care or even think about this stuff.
Does that mean I, as someone who has projects on PyPI, shouldn't care if I am unable to provide any attestations because I don't want to have to go through one of the recognized forges?
> Authenticity and minimally-scoped credentials should be ambient properties of a system
But if the only way to get them is to go through one of the recognized forges, then they're not "ambient properties" of just PyPI alone, because there's no way for me to use them just with PyPI alone.
Okay, but if the information is on the project page then it's pretty likely that users will look at this stuff without reading the docs. My feeling was the doc was written for people "Publishing Projects to PyPI" and not for people "Using PyPI", quoting the relevant sections of the documentation.
Of course, in real life 99.9% of people aren't looking at the page at all, and either following a documentation step which says "install by doing XYZ" or transitively from some other project they install.
I'll compare "attestations aren't something you even really enable" with this from the email attestation proposal: "Another possibility here would be to have it be opt-in: maintainers can mark their already-verified emails as being eligible for attestations on PyPI, e.g. via a checkbox in the user settings." ;)
I also can't help but wonder who reads https://docs.pypi.org/publishing/publishing-to-pypi/ where it says "For instructions on how to package your Python project, please refer to the Python Packaging User Guide.", follows those steps including the one about using twine and TestPyPI, does all that, and then gets to "Publishing package distribution releases using GitHub Actions CI/CD workflows" and gives up, since twine works.
Or goes back to the pypi documentation and sees "To use an API token: Set your username to __token__" when I can't find anything like that at https://packaging.python.org/en/latest/guides/publishing-pac... , Those seem like describing two very different systems, speaking as someone with no experience with them at all, who learned to use twine because of the Python Packaging User Guide.
I don't think we require a Trusted Publishing provider to use a physical HSM. As I've said in other venues, even a self-hosted Vault instance would almost certainly be fine. But storing private keys on disk is not, and we owe PyPI's users a higher standard of care than saying otherwise.
(I also don't hand out gold stars, and my employer does not direct my work on PyPI.)
Sounds like it. Currently it seems you'll need to setup your own OIDC provider[0], then work with PyPI to get them to trust your instance as a "Trusted Publisher". It seems like a system to scale for multi-tenant publishers (like millions of GitHub or GitLab or Google build or ActiveState users). The design itself should be able to scale for individual OIDC providers through webfinger and OIDC discovery like Tailscale does it[1], but it doesn't look like it's there yet. Not sure if it's planned either the github issue mentiones future PyPI verified email attestation, though I'm not sure if that implies webfinger/OIDC like ts does it.
> The design itself should be able to scale for individual OIDC providers through webfinger and OIDC discovery like Tailscale does it[1], but it doesn't look like it's there yet.
This is the right idea, but it's actually not that simple unfortunately: Trusted Publishing relies on per-IdP claims that are more precise than the `sub` claim (because doing pattern matching on the `sub` claim is brittle and prone to security problems). So we require publisher providers (OIDC IdPs) to provide distinct claims that we can match on. For GitHub and GitLab that's things like the repository name, workflow filename, etc., while for others it might be distinct username and immutable ID claims, etc.
This is all downstream of OIDC-driven federation being a much weaker and more bespoke process than all parties would like it to be. I think the discourse around this could be much improved by people taking the time to understand these complexities and why they pose a challenge to groups with limited engineering resources (like PyPI).
Provenance attestation is about security compliance, not real security. At a big organisation, you can check off compliance check-boxes because you're using Microsoft's big hosted service.
For regular developers, provenance makes no difference; whether the package was made in a box controlled by Microsoft with a proprietary stack or on a box controlled by the developer make no difference.
Not directly the subject but they switched from Material for MkDocs to Zensical.
It’s quite sad to see that one of the main (or the biggest?) documentation system for the Python completely alienated its user base and led to this fork.
Homebrew actually has full provenance. We added it in 2024 (I think, that’s from memory). We never flipped the default verification policy because of some integration concerns, but my understanding is that various downstreams and MDMs do consume and verify Homebrew’s attestations.
Edit: I should qualify: Homebrew’s own processes verify its attestations. But that’s not enabled by default on end user machines because it slows installations down and currently requires the `gh` CLI. Neither of those is a tenable long term constraint for a default feature.
let me tell you something incredibly important about the attestations / trusted publisher thing:
if you have any of your files published by a trusted publisher with attestations, then you must have all of your files published that way (in that package).
otherwise pypi determines your project may have been compromised and puts some big red warnings up like [1]. it does not try to look at "this particular source file was never part of trusted publishing" or anything like that. and, CRITICALLY, it does not send you an email or anything like that about the non-acceptable file that you just uploaded, which it REALLY should. I mean the model is, they think these uploads are malicious by an attacker. Emailing the project maintainer(s) would be pretty important! (When pypi wanted to get rid of GPG signatures, they started sending me an email for everysinglesignature that got uploaded. so i know you guys know how to send emails).
it's also AFAICT a permanent switch. once you put a single trusted/attested file up there, your account must now do all files in this way forever (or risk big red warnings. which maybe you're OK with that).
It would all be easier to take if trusted publishers weren't limited to solely large for-profit corporations (I'm too lazy to research if they're trying to fix this).
> it's also AFAICT a permanent switch. once you put a single trusted/attested file up there, your account must now do all files in this way forever (or risk big red warnings. which maybe you're OK with that).
It's not[1].
> It would all be easier to take if trusted publishers weren't limited to solely large for-profit corporations (I'm too lazy to research if they're trying to fix this).
As mentioned in several other places, we're always interested in adding more publisher providers, provided they meet our key management and notability requirements.
(This is also the easiest way to reason abductively about PyPI's goals. If PyPI's goal was to support "large for-profit corporations," we'd have dozens of publisher providers of dubious quality and community value, rather than the small subset we currently have.)
> This is also the easiest way to reason abductively about PyPI's goals. If PyPI's goal was to support "large for-profit corporations," we'd have dozens of publisher providers of dubious quality and community value, rather than the small subset we currently have.)
I never claimed pypi is trying to "support large for-profit corporations". I fully understand the "large corporations as trusted publisher" thing is because this particular subset, e.g. github, gitlab, google, are widely known institutions with "good" reputations (in this area). The issue is that pypi is not, as of yet, supporting individual projects that want trusted publishing without relying on a large for-profit corporation. I'm not too put off by it but it is definitely a thing that a private for-profit host overnight becomes horrible (see e.g. bitbucket) and I have to get off that company ASAP. Having some non-profit entities here would be extremely helpful.
> Having some non-profit entities here would be extremely helpful.
I fully agree. I am hopeful that Forgejo or whatever other underlying component of Codeberg can improve its key management primitives here, which would make it very easy to argue for their inclusion as a federated party.
Thought we already learned years ago an IP address is not a person?
That is to say; what the screen on my end communicates is not guaranteed to be representative of ground truth.
The IT world was recently making fun of Sam Altman's eye scanner because remote attestation can be faked.
All the layers of abstraction wrapped around the code is security through obscurity.
The code itself must still be verified as safe.
The cloud era has resulted in a lot of people operating at layers of abstraction so far removed from physics this shit has more in common with baseless dogmas, gospels, and spiritualities at this point.
We anthropomorphized the cloud inserting ourselves into it. It's just a bunch of networked computers.
You're speaking in such shorthand that I have no idea how to untangle what you're actually trying to say/propose. I suppose you're saying that attested code can still be malware because malicious actors can also go through the attestation process?
Yeah, this check basically just proves that you hosted your source code on a big platform and then built the pypi package with their CI. This is not particularly strong as a trust signal. Especially since it doesn't particularly protect against compromise of access to the code repository. I worry that this is going to distract from better approaches (like reproducable builds and/or distributed review).
Not very keen that the whole story of trust is basically centralisation - requiring "big cloud" and essentially promoting lock-in.
Did I get that right? Seems a bit anti-pythonic.
I believe you got that wrong.
Here's some extra context [0]: https://github.com/pypi/warehouse/issues/20927. Loooots of links to click in.
the point here being small service providers are _also_ able to get into this system, but I think there's an expectation of success.
The bar for trusted publishers is very high though. I don't think it's high "on purpose" to leave out certain platforms for being too small, but in effect if (for example) your provider doesn't have a security contact point then that's going to rule them out here.
I think at this point the group of devs+ who are not fully on board with control by and dependence on big tech/VC is a small niche unfortunately.
I don't think this is true. From my experience, developers are the people who care about this topic the most. Just look at HN, almost everyone (who speaks out) is against.
I'm very in favor of this work, but don't generally comment because these comment threads are full of folks strawmanning the work being done, and in my experience those folks aren't really interested in productive discussion.
I'd be wary of trying to estimate the split of opinions based on who comments.
You got that right. In this thread we have a ClosedAI employee promoting ClosedID services. Look at the sponsors of ClosedID:
https://openid.net/foundation/sponsoring-members/
We know why PyPI ripped out PGP support and has refused any verification mechanisms that other distributions use for decades.
I trust myself way more to with with a pgp key on my computer than figuring out how to use GitHub actions properly...
The problem is no one else does. Github workflow is at least public and auditable. Your personal computer is not.
So what? Ultimately the trust model is supposed to be there to validate that this latest blob of code is coming from the same source as the last blob of code for the same project, authenticated with the same public key. And that's all it is telling you, and all that it can tell you, whether the attestation is generated during a Github workflow or by a program running on my personal computer that you can't see. It is not telling you that this latest blob of code is safe and free from problems. It's just telling you that it comes from the same source.
The average person using GPG isn’t actually providing me much evidence in that regard, given the number of ways to have technically used a GPG key to sign an artifact that aren’t secure.
That's easy to handle: if the signature isn't secure, it doesn't count as an attestation. I have no problem with putting the onus on the person doing the signing to do it right. But at present doing it right on your own is not even an option. Only going through one of the tech giants is.
How is PyPI supposed to tell if the person signing the artifacts is handling their key securely?
They can't. But they can't tell for sure if large tech corporations are doing that either. Data breaches are a thing. Auditing Github workflows doesn't prevent them.
1. "ClosedAI" is not gonna win people over to your side, IMHO. Apple doesn't plant trees, Microsoft makes big hardware, and Python is not satanic.
2. I'm not entirely sure what the link is supposed to prove -- would appreciate a word of clarification if you find the time. It almost feels like a dogwhistle of some kind, but A) I doubt anyone on HN is anywhere near that problematic, and B) there's no clear pattern in those companies other than "interested in ID tech", and none of the usual dogwhistle targets made the list. Also, y'know, OpenAI isn't on it...?
I guess my fundamental confusion isn't about the list and more about what "we know". Are you implying that they are against pure, unmanaged PGP for this system because... I mean, why? Because they're playing the long game to promote a verification/attestation/identification (?) technology that you think offers less stringent privacy guarantees, and thus to sell more ads in 2036?
If so, doesn't that seem like a kinda roundabout way to raise Display Ads revenue, otherwise known as "that one dying industry"?
It's anti lots of things, but I don't think it's anti-Pythonic. Python is about pragmatism & empathy if it's about anything -- bitcoin and such don't factor into it. See: Astral/OpenAI, Anaconda, most of the PSF board historically being employees of big corporate users of python, NVidia's relation to the ecosystem in general, etc.
As a passionate anarchosyndicalist, I, for one, am thrilled about it. Decentralization is a tool in our toolbox for identifying and dismantling unjust hierarchies, but it is not a panacea, and can be quite damaging when taken to the extreme. See: cryptocurrency speedrunning 200y of bank regulations in 20y.
To put my trust for Python in more concrete terms: they support GitLab. As a passionate GitLab shill, I really couldn't be more impressed! It continues to shock me how infrequently the second most popular forge is acknowledged, much less supported. This shows they're not just, IDK, bowing to Microsoft's demands.
> This shows they're not just, IDK, bowing to Microsoft's demands.
Not only don't we, but I must emphasize that Microsoft did not pay for anything involving the implementation of either Trusted Publishing or attestations on PyPI. I wrote a personal blog post explaining some of my own rationale for why we prioritized GitHub and GitLab for PyPI's initial support targets, and it shouldn't be surprising to anybody who knows me that the reasons are pragmatic, not dogmatic[1].
[1]: https://blog.yossarian.net/2024/11/18/Security-means-securin...
Is "a public key that I personally upload to my PyPI account, using the credentials you already have that allow me to use that account" on the list of targets that will ever be supported?
If not--i.e., if only recognized "forges" are ever going to be supported--why not?
https://news.ycombinator.com/item?id=50499442 Seems relevant here.
Thanks, I'll comment further in that subthread.
All absolutely reasonable. At the same time though there are a lot of roads leading to centralisation, and as they all converge it gets just a bit .. problematic.
Absolutely not claiming its not a pragmatic step, but equal-footing alternatives would be useful.
I'm no tin-foil-hat - but when a website account requires government id (like in the uk), and creating software requires signing by the website, you link all the things - is that a good outcome?
This is true, and it’s why I’m hopeful that some of the newer forges can make meaningful default posture improvements that make migration away from centralized services the dispassionate choice to a user who cares about availability and security.
Thanks for the link - your blog is great! eg https://blog.yossarian.net/2026/04/11/Brocards-for-vulnerabi...
Some examples:
dlt (no provenance): https://pypi.org/project/dlt/#files
datasette (yes provenance): https://pypi.org/project/datasette/#files
The source distributions have icons while the wheels have not. Strange and confusing.
That's probably a mobile bug (I can't reproduce it on a desktop browser). Could you open an issue here[1]?
[1]: https://github.com/pypi/warehouse/issues/new?template=bug-re...
I see the icon for both, but in different places (below the name for the wheel, alongside for the source).
This is great! I want sigstore-based or at least sigstore-style "keyless" short lived signatures to become more and more popular, so that support gets better everywhere. In particular, if having secure OIDC handling in a CI/CD pipeline becomes something that keeps people on GHA or GitLab then I'd hope third-party hosted CI/CD services begin to take that gap seriously and work on meeting those requirements.
I'm bought in on keyless signing, and I want there to be many more providers, so I like to see more adoption which builds more momentum.
>I'd hope third-party hosted CI/CD services begin to take that gap seriously and work on meeting those requirements.
How is a third party service, especially a self-hosted one, expected to meet those requirements? They seem to be lengthy, onerous, and also require a detailed vetting by the pypi team that is unlikely to scale beyond a few large hosting providers.
(What's more, this does relatively little to actually signal trust to someone downloading them. All it does is provide a signature that one of the big providers has built the package from a given source (but nothing about the trustworthiness of that source). This is a problem much better solved by reproducable builds and distributed verification of those builds)
Tangled and Codeberg both seem to support issuing JWTs during actions runs.
Codeberg: https://codeberg.org/mfenniak/oidc-jwt Tangled: https://tangled.org/docs/spindles/workflows/#audience
I think OIDC tokens from workflows is basically table stakes at this point, since the alternative is a static secret, which is basically guaranteed to leak eventually.
All that's left is for PyPI to accept these tokens in their set of trusted publishers.
Please see the other comments in this thread (as well as on PyPI's issue tracker) for some important context here. It's not enough to throw JWTs over the wall, PyPI also needs to know that it can trust any given service to operationalize the ugly and risk-prone parts of managing PKI.
(I haven't looked at Tangled at all. If Tangled has a secret management story for its PKI that's stronger than a private key on disk and can demonstrate that a non-trivial number of PyPI users would benefit from federation, then it's a strong candidate.)
Sorry I didn't intend to insinuate that pypi should necessarily accept any old tokens just because they're there. Just that increasingly OIDC is more and more an option, which is generally a good thing.
PKI definitely is still hard, but if it can be done well by a smallish number of trusted parties to the benefit of their users, this can have huge ecosystem benefits. Let's hope we live in that timeline.
Very cool to see PyPI put more emphasis towards exposing provenance info. It would be even cooler if trusted publishing would finally support reusable workflows (https://github.com/pypi/warehouse/issues/11096).
It would be even nicer if there were a way to have an attestation without publishing with a workflow controlled by one of the tech giants. There's no way for me, someone who has had a PyPI account for many years and has used it for all that time to upload packages, to attach my own public key to my PyPI account and then sign uploaded distributions with my private key so the signature can be verified and count as an attestation.
I think the problem is this type of attestation is hard to weigh as a trust signal, vs the cloud CI version.
Inherently, if someone steals your key, or gains access to a box that can use that key for signing, they are indistinguishable from you. Hand PGP signing also lacks time-bound signing, any ratchet mechanism, CT log, consistent key hierarchy, or standard to bind signer name and project identity.
> this type of attestation is hard to weigh as a trust signal, vs the cloud CI version
In the sense that I'm not a large corporation and nobody else is auditing my personal computer, yes, that's true. Note, though, that large corporations are also subject to nefarious actors misusing their infrastructure. What if a rogue employee hacks their internal system? Or Russia or China manages to get a backdoor into it?
In any case, all this doesn't make the method I described impossible. It just means that attestations produced by an individual workflow need to be clearly labeled to distinguish them from attestations produced using a tech forge's workflow, so people will understand what, specifically, is being attested to. Then people could weigh each trust signal as they choose.
> Hand PGP signing
To be clear, I wasn't talking specifically about this method of signing with a personal key. I agree there are lots of issues with PGP as an overall system. I was only talking about the general idea of having some way for individuals to independently generate an attestation without having to go through a tech forge's workflow.
If only there had been PGP signatures. Oh wait ...
Maybe they'll implement PGoogleP, where your private key is safely and securely stored on Google's servers
This is about detecting formerly benign projects that have had their pypi entries hacked, but not their private keys. It does not detect malicious projects.
And about detecting projects which dared to move their release builds away from GitHub.com/GitLab.com.
Am I correct in my understanding that the only trusted publishers are for-profit US companies? And these all use proprietary implementations which cannot easily be replicated elsewhere?
Like, if I want to host on Codeberg, or Sourcehut, what is the process needed for those platforms to become a trusted publisher?
As it happens, I build my releases on my own hardware using a pypa/manylinux release. Can I provide attestation, or am I required to use a third-party service to get that "x Without provenance" removed from my entry?
Support for Codeberg is being discussed here[1]. I think there's broad consensus among PyPI's maintainers and admins that more publisher providers are desired, but we also need them to demonstrate strong key management and rotation controls due to the degree of trust and access that becoming a publisher provider grants them (effectively, they become the PyPI credential broker for the projects that use their service).
(If you pan through that thread, you'll note that one of open issues with Codeberg is that their underlying server wants to store OIDC private keys on disk at the moment. This is okay for small deployments of personal services, but not for something that PyPI needs to federate with and trust.)
[1]: https://github.com/pypi/warehouse/issues/19786
So I, as someone who does my builds locally, can never have attested builds?
Will PyPI be presenting warnings or statements of caution which might cause users to be concerned about using my distributions? Or imply that my build practices are somehow inferior to depending on a third-party build system?
Yes, that is right - that's the purpose of attesting builds.
Do you see any other alternative?
The current formats (publish and SLSA provenance) are for attesting publishing and build steps, respectively, but there's nothing stopping PyPI from adding a different, new attestation statement for local builds (something that conveys "I did this locally, and I was this principal (email address) when I did it").
So are there any plans for PyPI to do this?
If I understand this correctly the fundamental problem here is what codesigning / pgp solve.
Attestations are code-signing. PyPI once supported PGP signatures on uploads, but as since disabled it because (1) only a tiny minority of users ever signed, and (2) many of the signatures and corresponding keys were either functionally insecure or unusable.
Both of these are latent problems when trying to use PGP in packaging ecosystems; the subset of ecosystems that successfully adopt PGP tend to have very different topologies (namely, small rings of trusted...publishers) rather than an "anyone can publish" architecture like PyPI has.
There's a discussion thread for email attestations here[1]. Those would support local builds.
But to take a step back: if you want attested builds because you think attestation is itself a gold star, then you're working from the wrong priors. Attestations do not imply that a package is secure any more than a hash does; what they are is a cryptographic signature over a set of claims, which allows for authenticated auditing of a project's state over time.
I think PyPI's documentation does a good job of conveying this nuance[2], and that much of the opprobrium of attestations comes from people either not understanding that nuance or imputing conspiracies into it.
[1]: https://github.com/pypi/warehouse/issues/18007
[2]: https://docs.pypi.org/attestations/security-model/#trustwort...
> what they are is a cryptographic signature over a set of claims, which allows for authenticated auditing of a project's state over time.
Which is great--but does not in any way require that the attestation can only be produced by working through one of a small set of recognized "forges". You could do this just as easily by, for example, letting me upload to my PyPI account a public key (the same as forges like Github do to allow SSH access), and then letting me sign builds that I upload using the corresponding private key. But that kind of workflow is not being supported. Why not?
> Which is great--but does not in any way require that the attestation can only be produced by working through one of a small set of recognized "forges"
That's true, there's some context elsewhere in these threads about how PyPI can enable email attestations. The short version is that PyPI wanted to make this work by default, which was trivial with Trusted Publishing (since it already provides a strong source of verifiable machine identity).
"Why not a public key?" is a good question, and the very short answer is that greenfield signing designs should not assume that all users are able to manage the secrecy of long-lived key materials, much less force them to make parameter decisions when (re)generating materials. This is the same mistake that the Web PKI made for decades, and has spent the last 15 years slowly untangling. The ideal setup is one where users demonstrate control over an identity, and then the signing scheme binds that identity to a short-lived signing key. That's how email attestations work.
> email attestations
From what I can gather from the github issue that was linked to, those would still require the email identity to be vouched for by a provider. (Though if whoever already is hosting my email could be a "provider", that would at least greatly expand the list of providers from the current small number of forges.) There doesn't seem to be any way for me to, for example, link a public key I generate myself to an email address that PyPI already has on file for me.
Have you considered that perhaps you are working from the wrong priors?
I believe your prior is that you expect people who read the PyPI page to understand what attestation means. I believe your prior is they will see a notice about "not from a trusted publisher" and not treat it like demerit or warning. I believe your prior is that seeing this message will get people to do more due diligence than they currently do, and that the due diligence will be relevant and appropriate.
Based on my experience with my users, I don't have the same confidence.
I don't understand what "authenticated auditing" means, so why should they?
What project state do people think is important to audit? My code repo is private, hosted on Sourcehut. My build system is private, done locally. The only thing I release is the pre-built binary package. There's very little state to examine, and I can't figure out why anyone cares.
I also have customers who buy a source code license. They have never asked for an audit of the project's state over time.
I read the "Security model and considerations" document. I found it opaque, no doubt because the entire topic is foreign to me, but it doesn't do a good job of explaining. Take the second paragraph:
"The most basic type of attestation (the type enabled by default in pypa/gh-action-pypi-publish) asserts that the project was published by an authorized publisher such as a particular CI provider in a particular code repository.".
I know pypa/gh-action-pypi-publish is some sort of GitHub thing. I don't use GH or third-party CI so that clarification does not help.
It doesn't describe what "authorized publisher" means, other than a generic example of "particular CI provider in a particular code repository". That phrase exists only once in the document. Is it different from a "Trusted Publisher"?
Why are the terms "authorized" and "Trusted" used, instead of something like "affiliated"? If someone is not a trusted publisher or an authorized publisher it's is too easily misinterpreted as "untrusted publisher" or "unauthorized publisher", which seems scary. If someone is not an affiliated publisher, "unaffiliated publisher" sounds less scary. (I'm not saying "affiliated" is the right word, only an example of a roughly appropriate word where the negation of it is not so scary.)
Then later, "It protects against modification of a project after it was built".
I think that's supposed to be "modification of a release after it was built"?
My use of "project" matches the one at https://docs.pypi.org/using-pypi/pypi-basics/ "A project is a collection of releases and files, and information about them. ... A "release" on PyPI is a specific version of a project." I want to modify the project with updated metadata as things change, and to make new releases.
Oh, and I can't find a description of what "build" means in that document. It's used as if everyone expects to know what it means, but a few years ago I learned that modern Python packaging uses "build" different than I do.
When I compile my Python/C extension I "build" it. When I make a wheel I "package" it.
I believe that modern packaging calls the whole thing "build"? Or just the latter step?
Incidentally, "python -m build --sdist" on my project does not work as a replacement for "python setup.py sdist" because my setup.py includes a code generation step to get from a clean repo to the Python/C extension files included in the distribution. I haven't figured out the modern way to handle that. I think it's because of the differences in philosophy of what "build" means, and what actions are allowed during different stages of the distribution process, which is why I think "build" needs a specific definition, just like "project" and "release" have specific definitions.
I mean, my actual prior here is that nobody should care or even think about this stuff. Authenticity and minimally-scoped credentials should be ambient properties of a system, not something that users even need to read docs for, much less enable. But if we're engaging with the premise that users are looking at this stuff at all, then I think it's facially reasonable to expect them to read the docs.
For context: attestations aren't something you even really enable. They just happen, by default, if you use Trusted Publishing from a forge like GitHub or GitLab (and hopefully, more in the future).
FWIW, your feedback on the docs is very helpful. It's genuinely useful to hear that the phrasing in it is imprecise, and I'll try and improve some of it this week. The main confusion that I think you're having (which is IMO reasonable) is that the doc was written to be theoretically splittable from the "Trusted Publishing" concept, i.e. so that "publishers" (principals who can publish) who aren't tied to a Trusted Publisher can attest to their published files.
> nobody should care or even think about this stuff.
Does that mean I, as someone who has projects on PyPI, shouldn't care if I am unable to provide any attestations because I don't want to have to go through one of the recognized forges?
> Authenticity and minimally-scoped credentials should be ambient properties of a system
But if the only way to get them is to go through one of the recognized forges, then they're not "ambient properties" of just PyPI alone, because there's no way for me to use them just with PyPI alone.
Okay, but if the information is on the project page then it's pretty likely that users will look at this stuff without reading the docs. My feeling was the doc was written for people "Publishing Projects to PyPI" and not for people "Using PyPI", quoting the relevant sections of the documentation.
Of course, in real life 99.9% of people aren't looking at the page at all, and either following a documentation step which says "install by doing XYZ" or transitively from some other project they install.
I'll compare "attestations aren't something you even really enable" with this from the email attestation proposal: "Another possibility here would be to have it be opt-in: maintainers can mark their already-verified emails as being eligible for attestations on PyPI, e.g. via a checkbox in the user settings." ;)
I also can't help but wonder who reads https://docs.pypi.org/publishing/publishing-to-pypi/ where it says "For instructions on how to package your Python project, please refer to the Python Packaging User Guide.", follows those steps including the one about using twine and TestPyPI, does all that, and then gets to "Publishing package distribution releases using GitHub Actions CI/CD workflows" and gives up, since twine works.
Or goes back to the pypi documentation and sees "To use an API token: Set your username to __token__" when I can't find anything like that at https://packaging.python.org/en/latest/guides/publishing-pac... , Those seem like describing two very different systems, speaking as someone with no experience with them at all, who learned to use twine because of the Python Packaging User Guide.
Of course: In principle we'd welcome a change, but there are technical difficulties in perpetuity.
Storing your private keys somewhere other than a regular file on disk is not a perpetual technical difficulty. Please assume good faith.
Where's the good faith in demanding a non-profit buys a $10k+ HSM just to get a gold star from an OpenAI employee?
I don't think we require a Trusted Publishing provider to use a physical HSM. As I've said in other venues, even a self-hosted Vault instance would almost certainly be fine. But storing private keys on disk is not, and we owe PyPI's users a higher standard of care than saying otherwise.
(I also don't hand out gold stars, and my employer does not direct my work on PyPI.)
Sounds like it. Currently it seems you'll need to setup your own OIDC provider[0], then work with PyPI to get them to trust your instance as a "Trusted Publisher". It seems like a system to scale for multi-tenant publishers (like millions of GitHub or GitLab or Google build or ActiveState users). The design itself should be able to scale for individual OIDC providers through webfinger and OIDC discovery like Tailscale does it[1], but it doesn't look like it's there yet. Not sure if it's planned either the github issue mentiones future PyPI verified email attestation, though I'm not sure if that implies webfinger/OIDC like ts does it.
[0] https://docs.pypi.org/trusted-publishers/
[1] https://tailscale.com/docs/integrations/identity/custom-oidc
> The design itself should be able to scale for individual OIDC providers through webfinger and OIDC discovery like Tailscale does it[1], but it doesn't look like it's there yet.
This is the right idea, but it's actually not that simple unfortunately: Trusted Publishing relies on per-IdP claims that are more precise than the `sub` claim (because doing pattern matching on the `sub` claim is brittle and prone to security problems). So we require publisher providers (OIDC IdPs) to provide distinct claims that we can match on. For GitHub and GitLab that's things like the repository name, workflow filename, etc., while for others it might be distinct username and immutable ID claims, etc.
This is all downstream of OIDC-driven federation being a much weaker and more bespoke process than all parties would like it to be. I think the discourse around this could be much improved by people taking the time to understand these complexities and why they pose a challenge to groups with limited engineering resources (like PyPI).
Provenance attestation is about security compliance, not real security. At a big organisation, you can check off compliance check-boxes because you're using Microsoft's big hosted service.
For regular developers, provenance makes no difference; whether the package was made in a box controlled by Microsoft with a proprietary stack or on a box controlled by the developer make no difference.
Not directly the subject but they switched from Material for MkDocs to Zensical.
It’s quite sad to see that one of the main (or the biggest?) documentation system for the Python completely alienated its user base and led to this fork.
Judging by the readme and homepage, there were probably some large differences brewing. Can't say I'm a fan of how it looks like they'll take the UI
Great news! Would love to see provenance on Homebrew (brew) as well.
Homebrew actually has full provenance. We added it in 2024 (I think, that’s from memory). We never flipped the default verification policy because of some integration concerns, but my understanding is that various downstreams and MDMs do consume and verify Homebrew’s attestations.
Edit: I should qualify: Homebrew’s own processes verify its attestations. But that’s not enabled by default on end user machines because it slows installations down and currently requires the `gh` CLI. Neither of those is a tenable long term constraint for a default feature.
For curious readers, to turn it on:
brew install gh echo HOMEBREW_VERIFY_ATTESTATIONS=1 >> $HOME/.zshrc brew verify gh
Downloading attestations will be quicker if you are logged in to github: gh auth login gh auth status
For a box you don't want full login privs on, you can use a GH PAT token with minimal privs.
If too slow/GitHub down:
HOMEBREW_NO_VERIFY_ATTESTATIONS=1 brew upgrade
https://docs.brew.sh/Homebrew-Security-and-Supply-Chain
let me tell you something incredibly important about the attestations / trusted publisher thing:
if you have any of your files published by a trusted publisher with attestations, then you must have all of your files published that way (in that package).
otherwise pypi determines your project may have been compromised and puts some big red warnings up like [1]. it does not try to look at "this particular source file was never part of trusted publishing" or anything like that. and, CRITICALLY, it does not send you an email or anything like that about the non-acceptable file that you just uploaded, which it REALLY should. I mean the model is, they think these uploads are malicious by an attacker. Emailing the project maintainer(s) would be pretty important! (When pypi wanted to get rid of GPG signatures, they started sending me an email for every single signature that got uploaded. so i know you guys know how to send emails).
it's also AFAICT a permanent switch. once you put a single trusted/attested file up there, your account must now do all files in this way forever (or risk big red warnings. which maybe you're OK with that).
It would all be easier to take if trusted publishers weren't limited to solely large for-profit corporations (I'm too lazy to research if they're trying to fix this).
[1] https://fosstodon.org/@zzzeek/117554513935692558
> it's also AFAICT a permanent switch. once you put a single trusted/attested file up there, your account must now do all files in this way forever (or risk big red warnings. which maybe you're OK with that).
It's not[1].
> It would all be easier to take if trusted publishers weren't limited to solely large for-profit corporations (I'm too lazy to research if they're trying to fix this).
As mentioned in several other places, we're always interested in adding more publisher providers, provided they meet our key management and notability requirements.
(This is also the easiest way to reason abductively about PyPI's goals. If PyPI's goal was to support "large for-profit corporations," we'd have dozens of publisher providers of dubious quality and community value, rather than the small subset we currently have.)
[1]: https://discuss.python.org/t/pypis-new-security-indicators-a...
> This is also the easiest way to reason abductively about PyPI's goals. If PyPI's goal was to support "large for-profit corporations," we'd have dozens of publisher providers of dubious quality and community value, rather than the small subset we currently have.)
I never claimed pypi is trying to "support large for-profit corporations". I fully understand the "large corporations as trusted publisher" thing is because this particular subset, e.g. github, gitlab, google, are widely known institutions with "good" reputations (in this area). The issue is that pypi is not, as of yet, supporting individual projects that want trusted publishing without relying on a large for-profit corporation. I'm not too put off by it but it is definitely a thing that a private for-profit host overnight becomes horrible (see e.g. bitbucket) and I have to get off that company ASAP. Having some non-profit entities here would be extremely helpful.
> Having some non-profit entities here would be extremely helpful.
I fully agree. I am hopeful that Forgejo or whatever other underlying component of Codeberg can improve its key management primitives here, which would make it very easy to argue for their inclusion as a federated party.
You want the ones that employ core devs through Microsoft or are approved by OpenAI, which codeberg.org is not.
This should be the top comment. PyPI as usual screws up and spreads misinformation about projects.
> https://blog.pypi.org/posts/2024-11-14-pypi-now-supports-dig...
Thought we already learned years ago an IP address is not a person?
That is to say; what the screen on my end communicates is not guaranteed to be representative of ground truth.
The IT world was recently making fun of Sam Altman's eye scanner because remote attestation can be faked.
All the layers of abstraction wrapped around the code is security through obscurity.
The code itself must still be verified as safe.
The cloud era has resulted in a lot of people operating at layers of abstraction so far removed from physics this shit has more in common with baseless dogmas, gospels, and spiritualities at this point.
We anthropomorphized the cloud inserting ourselves into it. It's just a bunch of networked computers.
All this stuff is a spoon that doesn't exist.
You're speaking in such shorthand that I have no idea how to untangle what you're actually trying to say/propose. I suppose you're saying that attested code can still be malware because malicious actors can also go through the attestation process?
Yeah, this check basically just proves that you hosted your source code on a big platform and then built the pypi package with their CI. This is not particularly strong as a trust signal. Especially since it doesn't particularly protect against compromise of access to the code repository. I worry that this is going to distract from better approaches (like reproducable builds and/or distributed review).
Nobody is looking at these pages
True, therefore downvoted.