> It's especially sensitive for a VC-backed startup that is measured thoroughly by GitHub activity, but we have to pull the trigger:
This sentence also illustrates the absurdity of this investment model. It imposes a trade-off between building good software, and complying with the investor's metrics. They probably call such metrics evidence-based, but this example shows that they arbitrarily capture some numbers to obscure the lack of meaningful measurements.
This has a security implication which is overlooked. Contributors to a repository have higher rights, such as avoiding approval requirements for fork PR runs. GitHub warns in the docs:
> When requiring approvals only for first-time contributors (the first two settings), a user that has had any commit or pull request merged into the repository will not require approval. A malicious user could meet this requirement by getting a simple typo or other innocuous change accepted by a maintainer, either as part of a pull request they have authored or as part of another user's pull request.
fair point! We believe "Require approval for all external contributors" should be a default setting, as you cannot trust anyone who is not a member of the organization
Actions runs from external contributors aren't run with Actions secrets; if you are using Actions right (i.e. not using pull_request_target wrong) you don't need to trust external contributors. (eta: iirc the original point of the Actions approval flow was preventing cryptomining spam from abusing free compute)
you can't trust org members either I have seen projects have inter maintainer fallouts. In general trust doesn't exist.
If companies can screw you over and claim it's a mistake, there isn't much a person can do.
It's all about level's of trust, a maintainer going rogue is less likely, a past contributor going rogue more likely but not too much, a stranger with a typo pr merged even more likely but still, a complete stranger least trust worthy.
Interesting approach. We’ve seen similar spam/noise problems appear in financial workflow automation too — especially when AI-generated submissions scale faster than manual review processes.
This is the correct assessment. This is not up to the open source community or individual projects to "figure out", any more than its up to me to figure out how not to get spam email.
Yeah well, our corporate overlords have decided that you're going to take your slop whether you want it or not, so its very much up to us to figure out. Capitalism isn't going to jump off the disaster train any time soon
We’re currently working on a feature that lets admins archive PRs. The goal is to give maintainers more control over how they manage contributions in their repositories. Archived PRs would be visible to admins only, so maintainers still have access to contributor history for auditing purposes and to meet any organizational or compliance requirements. Would this be helpful for you?
Your suggestion would help a bit but I would prefer the opposite: before someone can 'pollute' my pull request space and draw attention from subscribers I would prefer an acceptance step (just like a moderator on a forum) instead of having to archive the PRs.
This is especially important as (AI) spam increases and just because I am away for a few days or weeks I don't want those PRs lurking around.
What is the benefit of deleting a PR over just closing it? It seems like closing has the benefit of signaling what kinds of PRs aren't acceptable, which deleting would lose.
This. But OP wanted special requirements to open a PR. I.e. if those requirements are not met the PR is never visible to all and so admins can reject spam PRs without giving them a platform.
PR spam is a major problems for repo that run bounties. Maybe GitHub should temporarily block accounts from raising PRs if like 95%+ of them are getting rejected.
I feel like GitHub should have a system where you can give out tokens that are valid for e.g. 1 PR. If someone shows to engage in meaningful discussion and has a good idea to address an issue/feature, you initially give them one PR token. If the PR is of good quality, you can give them a few more, until they are contributors that can just create PRs as they like.
A similar system would be nice for issues, though I'm not sure what it'd look like if issues are the springboard for contributing PRs.
Not likely to ever happen (as others said), GitHub/MS want to sell CoPilot subscriptions/tokens and LLM-generated PRs are a part of that business model.
You could use a "OTP" to provide this: give out tokens ("OTP"), anyone with that token can submit, keep a record of the user (eg github username, email address) and token, run a bot to delete submissions that don't have a valid token, check the token+user pair, if there is a mismatch blacklist the user that token was given to whilst removing the PR.
Which is not chrome and still has ads...(Ironically).
The issue here is the core model is broken (misaligned incentives). That's not something you are going to fix with a github "downstream". A token system could help but it's easy to imagine ways that could be gamed, if not implemented well.
If search ads are blocked on search engines, then there is no revenue for the browser. It's that simple (on top of that Brave has other revenues, but the majority is search ads).
So it's a game of hoping that the majority won't change the default.
This is the main reason Brave does not block search ads specifically by default, but still block the other ads.
Blocking the other ads there are no consequences, since anyway this revenue is not shared back to the browser.
This is why the business model of Brave is cynical.
-> It's the same model as AdBlock and the "Acceptable Ads" (block all ads, except the acceptable ads, unless you disallow them)
It seems like some better basic metrics should be made front and center with PRs in this day and age. Yes AI is the driving force behind the current crop of problems but there are other issues. Yes it’s accessible if you go look but the point is people don’t have time.
the rate of comits/PRs total
The rate of PRs to repos they don’t own
The reject rate of PRs
The number of ban
An estimated “AI” or bot score or status flag
There are a few better attempts at GitHub metrics calculators but I have not seen any that move beyond the paradigm of more vomits is default assumed good. It’s time to foreground quality not just quantity. The GitHub “4 kpis” are entirely action oriented.
I've gotten tons of spam on repos that were purely ml research code. Things I saw copy pasted over hundreds of repos.
> Maybe GitHub should temporarily block accounts from raising PRs if like 95%+ of them are getting rejected.
It's so bad I'd be okay with a lower bar where it's flagged if they're posting the same message over multiple repos... FFS they aren't even stopping this shit https://news.ycombinator.com/item?id=47964617
This is what we get for telling everyone how amazing AI is at writing code. It started with the people selling AI and for some reason tons of independent developers, some quite well respected in our field, piled on. Facebook now laying people off and saying it's because AI is just so good adds more fuel to the fire. Now you have a bunch of people fully confident that their AI friend is pumping out amazing code and submitting it to projects that are completely overwhelmed
We made "Github contributions" a metric for people applying for dev jobs. So, of course, because devs are the kind of people we are, they started working out how to game that metric.
Some folks decided to start paying bounties on bug fixes, features, etc. Those bounties are fairly trivial by western standards, but are significant for developing countries. This creates a new career for developers; racing to collect the bounties on offer.
LLMs have exacerbated these problems by allowing existing people doing this to do it faster, and also allowing more people to pretend to be software developers and get in on the action.
If we stopped allowing LLM-authored contributions we'd still have too many shitty PRs. It would just be back to pre-LLM levels of "too many".
The answer is to make Github contributions valueless. Stop paying bounties, and stop using them to assess candidates.
> and for some reason tons of independent developers
Cowboy coders got a virtual cowgirl coder and sold it to everyone, hmm, maybe... (respected or not, solo devs don't always have the requisite skills to not be a cowboy, either due to lack of experience or lack of innate skill)
I don't know that I completely buy this narrative, though. There has been a strong, top-down push for this since the "beginning".
Makes me wonder if an ELO-based system would work to mitigate these issues. People who merged PR successfully onto a project, that had real issues acknowledged, the quality of their responses measured by other users reactions or something, etc, multiplied possibly by the degree of importance of the project where their activity has been made. Won't be about human vs AI, but actual helpful effective being vs low effort/spammy contributions. Issues and PRs could be sorted and filtered by their ELO score. I'm saying ELO as analogy to "score based given the context", not really a 1:1 translation of the ELO system.
Negative score would be reports from other users because of spammy content or not acknowledged issues, with a middle ground of neutral score (+-0) or little positive score to issues or whatever with clear good intention, but couldn't reach a proper merged PR or were not issues (e.g. issue existed but wasn't the correct repo to be addressed, PR was good but needed other stuff to be implemented prior to it, maybe in the long run, etc)
ELO is shockingly easy to manipulate. For example there was a literal jail with a decent chess player in it. He created a pool of players who got great ELOs by beating him, then used them to boost his rating higher. Wash, rinse, and repeat.
Given any manipulatable scheme, AI will figure out how to manipulate it. For the OP, what happens if a single AI manages to get through to contributor? Then it starts elevating other AIs to contributor, and we're off again. There doesn't have to be a purpose to this. Trolls will troll, and trolls armed with AI bots can devote endless energy to doing so. The more you work to keep them out, the more fun it becomes for them.
I wish I had an answer for that problem. But I don't.
ELO is a bad fit because it requires competition between submitters; but if the idea is interpreted as “contributor karma score” or similar (not everyone’s familiar with the mathematical nature of ELO), then the way to close the loophole is to only consider voting inputs from the human project owner. This project chose to have people lie to a webform rather than lie to a git interface about using AI, so I don’t expect it will be particularly successful at inhibiting AI use by project-involved humans, but certainly it’ll squelch a lot of noise from unattended/passersby.
I think they were saying Elo system as kind of a general ranking system idea instead of the actual algorithm.
You could probably use some kind of pairwise ranking algorithm (like anything based on the Bradley-Terry model) to rate human vs. AI contributions, but that would take a lot of manual effort.
Google is using it to (supposedly) improve their searching algorithms. They give testers two different versions and ask them what's better.
fix this problem by make the rating value tied to some paid currency - a repo owner would have to pay for the PR, and that PR contributor will now have more currency than previously. In order to have said currency to pay, the repo owner would need to have contributed to another repo whose owner have currency.
The totality of someone's currency is their reputation.
Of course, now the decision becomes...who is the central currency issuer that creates it?
>what happens if a single AI manages to get through to contributor
Then they'll get removed by the humans? Its about cutting down work, not about eliminating the work entirely
The current approach removes about 99% of their overhead it would seem. If they have to do a few manual interventions here and there, that seems like a huge win overall
Some say the greatest compliment a mathematician can be paid is when a concept that bears their name is so ubiquitous that we no longer capitalise it. (E.g. Abel with the abelian group).
I have built something like this and in process of collecting the data.
Frontier users: 527,865
Light indexed: 527,865
Ready to queue: 9,083
Fast scores ready: 0
Activity events 24h: 30,266
Fast scores completed 24h: 19,123
Deep jobs completed 24h: 3,043
Fast-score ETA: n/a
Deep-hydrate ETA: 69h
Stale running jobs: 0
GitHub backpressure jobs: 19,113
High automation signals: 4,608
Medium automation signals: 1,327
Completed jobs: 74,714
Biggest challenge is Github's rate limits. At this pace it will take two more months to have 98% coverage. But after that the maintenance should be quite straight forward.
This would just hurt new users similar to how you are unable to comment on 90% of subreddits on Reddit as a new user, because you don't have enough karma points, or how on Stackoverflow your permissions are severely limited until you do certain jobs. The incentives aren't very good in systems like this. Bots can be made to easily game the system while regular users are discouraged from even participating.
Some kind of vouching or scoring might make sense to help qualify contributions and many people have suggested similar recently. If by "ELO-based system" you meant "some kind of scoring system (not based on Elo)".
The Elo rating system doesn't make sense in this context; it's designed around collecting zero sum game results for a given community of players and building a model around it.
The problem with that is that there are certain kinds of users that like to take control of community projects. And then they take control of more, and bigger ones.
There are a lot of political tricks that get used.
What is scary is that one of those kinds of users are malicious state actors. Like North Korea and Russia...
> It's not a contract job— it's our optional way of saying thank you to the community.
The writing style in their onboarding doc has common AI tells (in the quote: em dashes, “it’s not A, it’s B” sentence).
I can understand that, perhaps they want to fight fire with fire or don’t have time as they already say. Still, it all feels like inadequate half measures to me.
The entire post is clearly LLM generated. I get that a person clearly put together some thoughts, but prompting an LLM to 'turn this into a blog post' is the kind of low effort content I thought was not appropriate for HN.
At least bringing up the underlying method (restrict to contributors) has spawned the discussion about how that's probably a bad idea on the security side.
Is the solution to everything simply more catgirls [1]? Proof-of-work was, after all, about countering email spam. PR spam is but the latest in that long tradition.
Proof of work doesn’t work here same as it doesn’t work for email. The effort to mint a valid PoW is always going to put the legitimate user at a disadvantage, whatever the implementation is. Someone with an incentive to spam will always be able to do it faster, more efficiently than you.
You can’t submit a PR because your laptop is too slow? Rent some hash rate from someone, and now you’ve just made a system of paying botnet owners to be able to make a typo fix on a github repo. HashCash was never used in the real world for a reason, it sounds cute but the incentives are so insane as to only work in a vacuum where you assume everyone isn’t cheating.
> Someone with an incentive to spam will always be able to do it faster, more efficiently than you.
Sure, but looking at the cost to do it at scale is the wrong metric. I surely can't compete with a career spammer on emails-per-second or even emails-per-dollar, but I also don't need to.
It's more about the expected-value versus the cost. For example, my expected benefit from one email to my family is (while hard to quantify) hopefully much higher than a spammer's expected benefit of one spam email going out, which has a very small chance of leading to any amount of money. Attaching a CPU-churn cost per email is something I can ignore on my desktop, but they have to at least budget for it.
I'd also like to note that the win-condition isn't as extreme as making spam (or other "crimes") truly unprofitable, it just needs to be less profitable than other things the time/resources could be used for.
We really need to solve SPAM itself here, I think there may be a way to do it. I.e., the problem of spam is NtoN scaling connections. The network has never been able to solve that problem (exponential is the hardest). Limiting communication in terms of mesh networking may be the ultimate solution - bots can't get to you because they can't reach you.
What needs to be invented is a bridging protocol - some way to establish "legitimate" lines of communication over a network, while preserving (to some degree) privacy and decentralization. AI can only enter this network by being explicitly added to the channel, and thereby explicitly and easily blocked (and also solving the general SPAM issue once and for all).
Just like we did with IP addresses. If yours is blocked by Cloudflare, you can pay a botnet operator a few dollars to use theirs! You can even use your credit card through a mostly-legitimate website. It's very convenient.
Anubis is actually not a cat. The original Egyptian deity is a god of death, and has a canine head. Anime catgirls and dog girls can look similar at first glance.
Those are the only two options to finding quality candidates?
Try talking more about the meta of coding itself. Get into the developers head by _talking_ to them and understanding how they would approach and attack different problems. You can show them code and ask them what they would do differently / how they would go about implementing X-Y-Z. Just because you can write foobar doesn't mean you understand how to apply algorithms or w/e specific problems [your] team has. It's _far_ better to understand how they would solve a problem over their syntax anyway.
> If the email matches their GitHub account, GitHub links the commit to their profile and grants them contributor status.
When the article mentioned email matching, I was concerned that it would break down when a contributor's email address changes. (I have contributed to more than a few projects over the years, using email addresses that no longer exist.)
However, it looks like they're not using the email address recorded in the author's original git commit, but instead a GitHub-generated address whose unique parts are the GitHub user ID and username. That should survive authors changing their email addresses. It would still break down if a contributor loses access to their account and has to create a new one, but that's probably less common.
I think this is why signed commits are also supported. My first thought was that this probably doesn’t work with signed commits. But, maybe it does since they are listed as the commiter.
Yup. At the very least, the "big dogs" aka Github and Gitlab should allow you to "claim" an email address to an account and only link it up when the commit in question either directly got authored from the web UI or got cryptographically signed.
Hi HN community, I wanted to share our approach to reduce amount of AI slop PR's and issues in our repo. We enabled "require prior contribution" flag on GH and created a CI script that creates a tiny commit co-authored with you, if you pass captcha on our website. Worked really well and we were able to block at least 500 bots in the first week. Sharing a screenshot from cloudflare: https://archestra.ai/hn-comment-cloudflare-challenge-outcome...
Who do you add as a contributor though? Wannabe-contributors? Then they appear in the list of contributors before you even see if they're capable of producing an acceptable PR.
Your solution would be great if GitHub would also allow me to whitelist specific users, but unfortunately this still won't block "implementation plans" in comments.
I'm not sure why gh hasn't already implemented stricter measures / filters / tools for PRs. It would cut down on spam and also help save their servers that can't handle the increased AI load!
Repos get forked, code gets pushed, all before a PR is created. What kind of measures can be implemented to cut down on the AI-general forks and pushes?
Sounds kind of weird that the blog post complains about `poisoning the conversation with pointless "implementation plans"` when literally they ask for that, after attaching $900 USD bounty to a very under-specified issue, and even replies with "Do you have an implementation plan in mind?" to some of the first "attempters". Sounds like they got exactly what they'd been asking for, and even before LLMs if you pulled something similar, the effects would have been similar.
It's fine for developers to provide a plan, even if it gets rejected. The problem comes when every script kiddie figures AI has made them into a developer.
Imagine you want to get a doctor's opinion, or maybe a couple of opinions. But a zillion AI-amateurs have registered themselves as doctors. How do you separate wheat from the chaff?
> Imagine you want to get a doctor's opinion, or maybe a couple of opinions. But a zillion AI-amateurs have registered themselves as doctors. How do you separate wheat from the chaff?
Right, but that's not what happened though.
Someone went to the public square, said "Hey, I'm looking for any sort of doctor, and I'll pay you $900 if you tell me your plan and then whatever plan I chose wins" and then they get surprised they get flooded by zillion AI-amateurs.
You don't generate a ton of chaff then try to find the wheat, you ensure your process doesn't generate a ton of chaff in the first place. Offering large monetary rewards for relatively simple work for anyone in the public is bound to generate a ton of chaff...
This is great example of the toxic effect money has on open source. Reward people with respect and recognition instead. Weird anonymous accounts no one's ever heard of will leave, because someone (or something) who's concealing their identity has nothing to gain from recognition. Honestly GitHub should have a real names policy. Because if you're not Satoshi Nakamoto then there's only three reasons I can think of to be anonymous on GitHub: (1) to avoid obtaining your employer's authorization, (2) to spam, harass, and engage in toxic behaviors, or (3) you're not even human. All three of these are the last things I want when engaging on the GitHub platform. Don't get me wrong, I love robots. But I'm perfectly capable of talking to the robot on my own. I don't want to talk to your robot. I also don't want people slipping me intellectual property below the board without their employer's consent. And I certainly don't enjoy all the hate and harassment. GitHub has tried to help with the last part, by making overt displays of hate something that can get you in trouble. The issue is that people just get more guilesome with more anonymous accounts, because the issue was never disrespect (which can actually be strategic and pro-social if we look at Torvalds' career), but rather bad faith participation. If GitHub can guarantee that all its users are human real names good faith actors, then we might be able to start talking about open bounties.
> someone (or something) who's concealing their identity has nothing to gain from recognition
The xz supply chain attacker hid their real identity, created fakes one and gained recognition over time in order to gain more access and add the backdoor. So TLAs and other bad actors at least are interested in gaining recognition.
I know, right? It's like, finally—a threat actor who's intelligent enough to understand what capital means in the open source community and is willing to devote resources to engage with it authentically (even if it's for evil nefarious ends). The xz incident showed that the open source community has many other good defense mechanisms for verifying and spotting malicious work and then solving it. But we won't even get to play that game if we're inundated with anonymous agent spam so that GitHub can juice its MAU numbers. Maybe they should require every account buy a $40 yubikey. I don't know what the answer is. But I know that no one gains when your measure of success is driving the cost of burning open source developers out down to literally zero.
'I will take "problems that could be easily be solved by implementing a Pfand system" for $200, Alex.'
Seriously. Just ask for a US$10 deposit for the each PR. If the PR is accepted (not even merged, just accepted as "this is a good effort"), give it back. Hell, give double the amount for good effort and you got yourself a cheap way to attract good contributors.
Best case, bots will balk at the payment. Worst case, the funds can be used to hire someone specifically for triage.
This sounds like a great idea until you think about it for more than 30 seconds. Similar to most "it's so easy, you just" ideas.
Seriously, chill, then think about how you'd implement it. Then think how it'd go wrong. Then think about how to fix those problems. Repeat until you realize there's a better solution or until you solve the problem without making it overly convoluted. More often than not the former is the better option. More often the latter is just a variant of the sunk cost fallacy and your ego. Reality is (un)surprisingly complex and solutions aren't usually trivial
This is an overly negative response to a genuine solution. There are a million reasons you shouldn't do X or Y.
More than likely GitHub would have to maintain their own internal wallet solution for this, which is a big engineering lift. But we're all just having a discussion.
Except it isn't. It is a lazy solution and impractical one
> More than likely GitHub would have to maintain their own internal wallet solution
Great, so you even found one of the main issues, which pushes off the problem to a third party which makes it an impossible solution for anyone but GitHub (still a problematic "solution" though)
> This is an overly negative response
Yet it isn't because even as you noted it's not realistic to implement.
There's two types of lazy, and this is the kind that creates more work, not less
Then people from a sanctioned countries are blocked from open source, or worse, you have to explain to the bank and/or the government why you sent 20USD to someone in Venezuela.
I think the intersection of the set of people able and interested in contributing and those who are willing to figure out cryptocurrencies is the empty set.
I quick visit to https://gitcoin.co/campaigns will show you that you are wrong. Hundreds of projects funded by even more people.
Mind you: that's on one of the most convulated ways there is to get involved, because it involves a bunch of smart contract operations and on-chain voting. If we are talking about crypto only as a payment network, things are even simpler.
Coinbase reports more than 100+ user accounts worldwide. Kraken has ~10M.
Also, we are talking about people who are tech-savvy enough to be interested in participating in a FOSS project. Opening an account at an exchange is not rocket science.
No, getting the money out of an exchange if you are in a country USA doesn't like is rocket science. Which was the whole point of using crypto rather than money.
The exchanges are for the people that can send the money, not receive it.
And those who are living in countries the USA doesn't like will probably have no issue to learn how to work with crypto. Of all the complex things they need to do to work around the restrictions, setting up a wallet barely registers.
This is exactly the strategy that the owner of the SomethingAwful forum used in 2004 to get rid of bots and assholes. (I used to remember his name, kinda famous, oh well).
I have 3 PRs on https://github.com/django-oauth/django-oauth-toolkit/pulls that haven't been merged for OVER AN YEAR due to the maintaners being overloaded and who are expected to work on this for free. The fact that these PRs are not being promptly reviewed have cost me at least 3000€ in potential grant work.
If I was told that I could make a deposit of $10 to get less stressed maintainers and a faster PR review cycle, I wouldn't even blink. I wouldn't even ask for the money back.
This is an evergreen internet comment right here. Condescendingly proclaiming "This problem could be easily solved by [significantly worse solution that had 1/10th the thought put into it as the actual solution by people with a stake in actually solving the problem rather than making quippy armchair comments]".
---
I know it's against convention to comment on downvotes, but really? Really? This is controversial? The OP came up with an elegant solution that cleanly solved their problem without subjecting contributors to anything more than a captcha. Then somebody comes along and says "oh, it's so easy, just charge $10". You're going to set up payment infrastructure, incur administrative overhead with human support managing refunds, and deter 99% of actual humans from contributing, and then call that the easy solution that OP is so stupid for not thinking of first? Give me a fucking break. This site really is just Reddit-lite, anyone who thinks about engineering problems seriously would realise this does not stand up as anything beyond a pithy internet solution with three seconds of thought into what actually implementing it would entail.
Polar.sh is already doing things that are a lot more complex in this space.
If you are in a civilized country which allow direct payments (i.e, anything but North American nowadays) and you don't want to deal with Github or any external system, there is always good old "make a M-PESA/SEPA/Pix/UPI transfer to account XYZ")
> the thought put into it as the actual solution by people with a stake in actually solving the problem
Let me flip your argument: think of how much time and thought is poured into problems like this one by people who don't even try to implement a Pfand system beforehand.
...which is not available to maintainers to use in this way.
> there is always good old "make a M-PESA/SEPA/Pix/UPI transfer to account XYZ"
And then lock out anyone who is not from the same country as the maintainer, on a platform that is known for its global reach.
Moreover, you're introducing significant anti-human friction. For privacy-conscious people, it's a complete non-starter; I'm not giving my payment information, not for a $1 transaction, and compromising my anonymity just to make a PR for the benefit of other people. That's a small subset. Then, you have the lazy people. The majority of the population will simply not bother with something if it has friction. Getting out their credit card is one of those things, and it's why products/services that offer free trials or a free tier tend to be overwhelmingly more successful -- people want to see a tangible benefit to themselves before they engage in high-friction processes (where "high-friction" is as little friction as requiring a payment, yes). "Free to play" video games with microtransactions engineer first-time purchases to be cheap ($1 or $5) and have 5x or 10x the value of the normal microtransactions, because that first hurdle of getting somebody to hand over their payment information is by far the biggest.
I'll take the captcha, thanks. And maintainers will too, because they'd rather have the solution that filters bots and keeps humans contributing rather than the one that filters out both humans and bots.
Yes, that friction is intentional. The lazy people don't want to do it? Great, there is very little chance their contributions are worthwhile. The privacy conscious people won't do it? Then let them work on their own repositories and complain loudly about the idiot maintainer who puts these insane barriers. Then the maintainer can go take a look at that forks done by the loud complainers and see if it is worth to whitelist them.
> it's why products/services that offer free trials or a free tier tend to be overwhelmingly more successful
Drug dealers also offer the first hit for free, why don't you use that as an example as well? ;)
To answer this properly in case the quip was too vague: there is no reason for "number of PRs opened by new contributors" to be a viable/interesting KPI for any FOSS project.
> I'll take the captcha, thanks.
First you need to show me all your cool FOSS projects.
Oh, give me a break. No one is taking the ability from others to fork the repo. If these exceptional cases really were to happen, how fast would it be for someone else to notice and do one of (a) notify the maintainers to get this particular user whitelisted or (b) front the entry costs?
And a registration system that amounts to a more complicated captcha doesn't? How long until someone starts farming accounts and run bots that jump through these hoops as well?
> wonder if the idea is fundamentally broken.
It's only "fundamentally broken" if you need to build a perfect system that needs to accept 100% of legit PRs without raising any level of friction.
But we don't need that. Pfand systems are not meant to be perfect, and they are not meant to the single solution to any problem involving the commons. They will not get rid of all bad behavior, but they will certainly bring it a global-scale problem down to levels that can be then managed by other smaller, context-aware systems.
I semi-regularly offer drive-by PRs to projects I like and use. They're real PRs, not generated with AI. They range from papercuts to doc fixes to attempts to add that one feature that I want. Sometimes it's a drive-by group of PRs. Or an issue and a PR. I try to conform to what the maintainers prefer.
Unless I knew the maintainers personally, this would prevent most of my contributions, which are most often accepted. Maybe it's worth losing out on my small contributions to avoid slop. But things would absolutely be lost this way.
Agreed... However there's not a good IsThisAnAI() test at present. So unfortunately, we will have to use anti-spammer techniques (because that is exactly what AI is, high(er) quality spam).
I believe you are asking me in jest, but if you are genuine, this is what I would add to my CONTRIBUTING.md
```
# FIrst-time contributors
Due to the increased number of AI bots and low-effort contributions, we are being forced to add some friction for first-time contributors. PRs are closed for anyone not explicitly added to our list of authorized users.
To be accepted in the list, you must do one of the following:
- Show a history of meaningful contributions in projects from related technologies done before Jan 1st, 2023.
- Be vouched by one of the existing contributors in the core team
- (If you have github sponsors/polar/patreon) Be a sponsor for the project for the last 3 months)
- Submit a small payment, which will be held in escrow until your PR is accepted. The following methods are accepted (choose all that apply: paypal, SEPA, Crypto, Venmo, Pix, UPI, M-Pesa, etc)
```
Oh no, I'm being genuine. Documenting the process itself wasn't what I was curious about, I was wondering how you'd go about your last bullet. Accepting lots of currencies can be hard, but I guess I'm not super familiar with online escrow services. I'm not sure how simple they can make that process, or who would pay the cost of using them (I assume they're not free).
I was also wondering how automated or manual you would envision the review process. I'm guessing your hope would be that the small deposit would stem the flow of submissions enough to make it all possible to review manually again, and you would also manually return all the payments sent to escrow?
Yes, I'm assuming that adding requirement for payment would bring the number of requests down to a level which I could manage with a simple spreadsheet.
Paypal/SEPA transfers are free in Europe. And even if I lived in the US and had to pay a small processor fee, I'd be more than willing to cover the $0.50 in fees if that meant I was receiving contributions from people who went through all the trouble.
$10 to a Silicon Valley software engineer reading this comment may feel like... $(a lot?)... to a range of other would-be contributors (thinking of $6/day minimum wages in some places, for example)
Wonder if a dollar would work for now until more people give bots credit cards.
Easily? You think the kind of people who think it makes sense to make bogus slop PRs are going to react reasonably to overburdened volunteer maintainers refusing to give them their US$10 back?
This is never going to work. Sufficiently many of these people are going to find maintainers' home addresses and send them death threats and the likes. If you see how badly some people flip out just because their PR is rejected, it's going to be much much worse if their PR is rejected and their money is taken.
Ok. If I'm the maintainer receiving death threats over that, I'd tell them they would get the $10 dollars back, plus some extra money for their troubles.
Location of the envelope with the money: the same police station where I'd reported the death threats.
I don't think that is a (very realistic) concern. AI is slop, the problem is not that the real contributors are struggling to get PRs merged.
The bigger issue being, raising the bar to students who may have otherwise had productive careers (but education is a general issue, where the students don't even yet recognize they are being scammed).
I don't follow, and I'd be concerned that this opens up a cottage industry of bots generating plausible looking repositories that unwitting contributors would attempt to contribute to. We already know that bots are astroturfing repos to generate overinflated star counts. I'd say the least crap option here is to honeypot PR contributions from bots
You keep describing this as not a likely or realistic scenario. But why is the likelihood even of relevance here? The way to avoid the worst case i.e scammed of your money, is to not even put it on the table in the first place.
Ill thought out logic like your own. I think you are likely a bot at this point.
It's not likely, because that's not something that people are likely to do. Only a bot like yourself with a poor model of the world will do this type of thing. It will be amusing to see the AI bots trying to run the scam you are describing and then nobody will contribute to the fake projects... except other fake AI contributors.
Dude, you're claiming that there's no likelihood of people getting swindled out of their money by handing it over to strangers. So your reaction is to play the bot card? We're done. You're clearly not arguing in good faith here.
> people getting swindled out of their money by handing it over to strangers.
I think that OP is trying to say is that there is very little reason for a human to go through the trouble of contributing to a "plausible looking fake repo".
To get to the point that a repo starts to attract interest from other contributors, that project needs to have actual utility.
Who in their right mind would jump into opening a PR from projects they never used? And if the project does get used to the point that it attracts people interested in contribute to improve it, wouldn't it mean that we've achieved https://xkcd.com/810 ?
Let's also be realistic, everything that can be automated will. Even if that thing is worse off for it. There's a clear historic pattern of this. Companies and people love to be penny wise and pound foolish.
You also lose out on a lot of would-be PRs. By people who don't have the money, don't have trust, or have a visceral "fuck you" stance. There's a lot more reasons that this suggestion creates a gate that dissuades the people you want. I stand by that the solution is naïve, but you're welcome to give it a try on your projects. I'm sure it'll be effective at reducing a lot of spammers, but I'm also pretty convinced it'll come with a large false positive rate, which is invisible (giving you false confidence)
I am more concerned about the sustainability of the projects as a whole than trying to optimize the number of potential random PRs.
> you're welcome to give it a try on your projects.
It's not exactly the same, but in a way I'm already doing that with Communick. I'm running one of the few Matrix and Fediverse services where members must pay to have access. Up until last week, I was giving 14 days as a free trial period and no deposit/confirmed subcription required. But now because of AI bots, I dropped it and I am collecting payment info before activating any account.
If I were playing the startup playbook, that would be insane. It's already crazy to try to charge something that people are used to get for free; my conversion rates are already low, requiring credit card info will make them even lower.
In the end of the day, I don't care. At first I was really hoping this would be something profitable, right now I just keep it running because I can. Even with the small number of users Ithe servers get, I get enough to cover the better part of operational costs, I get to sharpen my devops skills and have a test lab to learn what is that people really want (i.e, they are willing to pay to have solved) vs what people claim to be a problem.
All in all, the Communick instances are not going to win any popularity contest, but my servers have been up and running without major issues or drama for more than 6 years, and that's a lot more than I can say for all the other servers that have come and gone because the admins tried to play the numbers game.
> trying to optimize the number of potential random PRs.
You're misrepresenting my comment. I didn't say at need to optimize, just consider. Don't strawman me here
You can't just hand wave them away as if this isn't an important factor. If you don't care about them at all I got a much much simpler solution: don't allow issues or PRs. Problem solved! But that's not a real solution either
> You can't just hand wave them away as if this isn't an important factor..
There are plenty of ways to indicate in the project that the Pfand is meant as one way to filter out bad actors, but it doesn't mean that it should be the only way to accept external contributions. You can find somewhere else on the thread where I listed some alternatives that can be used as well.
> If you don't care about them at all I got a much much simpler solution: don't allow issues or PRs.
Yes, and what is the problem with this solution? That's what many projects are doing and many more will do. They will close access to non-members and only accept someone new when they have some type of social proof. [0] And that is totally fine.
If I have to trust you to give me back my $10, I'm never contributing to your code. Ever.
If I have to trust GitHub to give me back my $10, frankly, I have more trust for a random person on the internet at this point.
Also, you glossed over my banana joke, but it did hold meaning[0]
> Yes, and what is the problem with [closing down PRs and Issues] solution?
Are you serious? I mean it is an acceptable solution but it's completely orthogonal to the one we've been discussing. I can see you're not serious. I was skeptical because the first comment, but thanks for making that clear now.
What? No. A PR is me giving my time to the project. I don't get anything out of it except the warm feeling of having helped out. If I have to pay money to submit a PR then I'm going to play video games instead.
Unfortunately, the issue is that time is not enough of a filter anymore. The time from machines is basically worthless compared to yours, so you need to give something else, and that something else needs to be something that shows you have actual skin in the game.
I don't have a better solution, unfortunately, but it doesn't seem seem to like the spam problem has been solved. It has just been moved from pull requests to commits:
Currently, more than 10% of all commits in the archestra repo are essentially noise (369 of 3521 commits), accounting for more than half of all commits in the last month (303 of 578 commits).
But maybe (probably) the amount of such commits will go down over time, compared to the growing amounts of AI slop
As those commits were made from our system they don't create any noise for us, as PR/issues/email notification do. We only include real people who could solve the captcha and their input is mostly valuable
so...they are manually re-setting the "interaction limits" over and over again, since they are only temporary?
why not use hooks to automatically reject issue comments / PRs etc. from users that didnt go through onboarding, rather than repurposing GH features that aren't really designed for that use (and are hence in danger of being changed someday)?
GH sends the email notification to all subscribers at the moment of posting a comment. There is no cooldown or a way to unsend the notification using hooks
> While GitHub reports massive metric growth — a substantial part of which is AI-generated — we as an open source project team have to do the heavy lifting of cleaning up AI slop from our repository and come up with esoteric workarounds to keep the level of legitimacy of our open source audience.
There's got to be a concept to differentiate the industry plants who start an "open source" project that has enough funding for a $900 bug bounty. They are speaking and developing in the language of corruption and they don't even know it. Of course you will receive AI bot spam, but unfortunately it will continue if you don't take a hard look in the mirror.
It's quite ironic to complain about AI slop in a piece that's quite clearly AI slop.
Soon there will be no more AI doomer comments. The bots will take over that job too.
---
I'm working for an open source company, and my God, are 95% of contributions useless.
There are really dumb ones where the bot writes 10 paragraphs about how he implemented the feature, but the entire changeset is adding one line to .gitignore or adding a CLAUDE.md file.
There are even worse ones where the bot submits 3000 lines of code that seemingly works, but you have to spend an hour to figure out why it doesn't work.
This is funny to me because the title on this submission currently refers to “Git's –author flag”, which is an extremely incorrect use of a dash. (The original article doesn’t make the mistake. Not sure if the error is from the submitter or from an HN title mangulation.)
That's a neat way to interface with GitHub's authentication system, but I don't see how they've solved the fundamental problem because their whitelisting process is just "click ok fine 10 times". Why won't the slop peddlers just do that too?
I think the point is to add a bit more friction to the process. You want to make it so that people can do it with minimal effort and an AI bot will give up. If you're in an arms race over AI commits and PRs, this is a decent middle ground to start from.
(Why there is a race for AI commits/PRs to projects is beyond me though...)
What I see is a solution for a problem that is self inflicted, meaning lumping contributors and generic internet users in the same workflow. In big projects, you have the core team, a handful of well known contributors, and everyone else.
I strongly prefer the git email model, where it’s often trivial to control the flow of changes proposal. GitHub does not have the same wealth of tools and versatility.
> It's especially sensitive for a VC-backed startup that is measured thoroughly by GitHub activity, but we have to pull the trigger:
This sentence also illustrates the absurdity of this investment model. It imposes a trade-off between building good software, and complying with the investor's metrics. They probably call such metrics evidence-based, but this example shows that they arbitrarily capture some numbers to obscure the lack of meaningful measurements.
This has a security implication which is overlooked. Contributors to a repository have higher rights, such as avoiding approval requirements for fork PR runs. GitHub warns in the docs:
> When requiring approvals only for first-time contributors (the first two settings), a user that has had any commit or pull request merged into the repository will not require approval. A malicious user could meet this requirement by getting a simple typo or other innocuous change accepted by a maintainer, either as part of a pull request they have authored or as part of another user's pull request.
fair point! We believe "Require approval for all external contributors" should be a default setting, as you cannot trust anyone who is not a member of the organization
Actions runs from external contributors aren't run with Actions secrets; if you are using Actions right (i.e. not using pull_request_target wrong) you don't need to trust external contributors. (eta: iirc the original point of the Actions approval flow was preventing cryptomining spam from abusing free compute)
you can't trust org members either I have seen projects have inter maintainer fallouts. In general trust doesn't exist.
If companies can screw you over and claim it's a mistake, there isn't much a person can do.
It's all about level's of trust, a maintainer going rogue is less likely, a past contributor going rogue more likely but not too much, a stranger with a typo pr merged even more likely but still, a complete stranger least trust worthy.
Interesting approach. We’ve seen similar spam/noise problems appear in financial workflow automation too — especially when AI-generated submissions scale faster than manual review processes.
too — especially
No it doesn't have security implications.
If you are insecure because someone has had one of their otherwise completely innocent PRs merged into your repo... you are insecure, period.
What you are describing is exactly a security implication.
Security isn't a binary "secure/insecure". You can be more or less secure than something.
Screw GitHub for letting this happen. If they implemented some very basic requirements to comment and open PRs we wouldn't be here.
Also please let us delete PRs just like we can delete issues.
This is the correct assessment. This is not up to the open source community or individual projects to "figure out", any more than its up to me to figure out how not to get spam email.
Yeah well, our corporate overlords have decided that you're going to take your slop whether you want it or not, so its very much up to us to figure out. Capitalism isn't going to jump off the disaster train any time soon
We’re currently working on a feature that lets admins archive PRs. The goal is to give maintainers more control over how they manage contributions in their repositories. Archived PRs would be visible to admins only, so maintainers still have access to contributor history for auditing purposes and to meet any organizational or compliance requirements. Would this be helpful for you?
Not OP but requested this feature since years.
Your suggestion would help a bit but I would prefer the opposite: before someone can 'pollute' my pull request space and draw attention from subscribers I would prefer an acceptance step (just like a moderator on a forum) instead of having to archive the PRs.
This is especially important as (AI) spam increases and just because I am away for a few days or weeks I don't want those PRs lurking around.
What is the benefit of deleting a PR over just closing it? It seems like closing has the benefit of signaling what kinds of PRs aren't acceptable, which deleting would lose.
Closing a PR or issue still makes it discoverable in PR/issue search results, as opposed to deleting an issue.
This. But OP wanted special requirements to open a PR. I.e. if those requirements are not met the PR is never visible to all and so admins can reject spam PRs without giving them a platform.
PR spam is a major problems for repo that run bounties. Maybe GitHub should temporarily block accounts from raising PRs if like 95%+ of them are getting rejected.
I feel like GitHub should have a system where you can give out tokens that are valid for e.g. 1 PR. If someone shows to engage in meaningful discussion and has a good idea to address an issue/feature, you initially give them one PR token. If the PR is of good quality, you can give them a few more, until they are contributors that can just create PRs as they like.
A similar system would be nice for issues, though I'm not sure what it'd look like if issues are the springboard for contributing PRs.
Not likely to ever happen (as others said), GitHub/MS want to sell CoPilot subscriptions/tokens and LLM-generated PRs are a part of that business model.
My community does something vaguely similar, where you get credit for having bugfix PRs merged, and it's deducted when you get feature PRs merged.
You could use a "OTP" to provide this: give out tokens ("OTP"), anyone with that token can submit, keep a record of the user (eg github username, email address) and token, run a bot to delete submissions that don't have a valid token, check the token+user pair, if there is a mismatch blacklist the user that token was given to whilst removing the PR.
If only there were a simple way to make tokens that weren't fungible and could be given to others
/s
GitHub has not incentive for blocking AI. It's like asking an ad company to build an adblocker into their browser.
It's called Brave
Which is not chrome and still has ads...(Ironically).
The issue here is the core model is broken (misaligned incentives). That's not something you are going to fix with a github "downstream". A token system could help but it's easy to imagine ways that could be gamed, if not implemented well.
Ads are the main business model of browsers.
If search ads are blocked on search engines, then there is no revenue for the browser. It's that simple (on top of that Brave has other revenues, but the majority is search ads).
So it's a game of hoping that the majority won't change the default.
This is the main reason Brave does not block search ads specifically by default, but still block the other ads. Blocking the other ads there are no consequences, since anyway this revenue is not shared back to the browser.
This is why the business model of Brave is cynical.
-> It's the same model as AdBlock and the "Acceptable Ads" (block all ads, except the acceptable ads, unless you disallow them)
It seems like some better basic metrics should be made front and center with PRs in this day and age. Yes AI is the driving force behind the current crop of problems but there are other issues. Yes it’s accessible if you go look but the point is people don’t have time.
the rate of comits/PRs total
The rate of PRs to repos they don’t own
The reject rate of PRs
The number of ban
An estimated “AI” or bot score or status flag
There are a few better attempts at GitHub metrics calculators but I have not seen any that move beyond the paradigm of more vomits is default assumed good. It’s time to foreground quality not just quantity. The GitHub “4 kpis” are entirely action oriented.
Problem is the bots can create any number of github accounts and continue spamming. Though this would be a good simple defense to start with.
I've gotten tons of spam on repos that were purely ml research code. Things I saw copy pasted over hundreds of repos.
It's so bad I'd be okay with a lower bar where it's flagged if they're posting the same message over multiple repos... FFS they aren't even stopping this shit https://news.ycombinator.com/item?id=47964617GitHub and Microsoft are actively contributing to the problem, why would they admit fault?
This is what we get for telling everyone how amazing AI is at writing code. It started with the people selling AI and for some reason tons of independent developers, some quite well respected in our field, piled on. Facebook now laying people off and saying it's because AI is just so good adds more fuel to the fire. Now you have a bunch of people fully confident that their AI friend is pumping out amazing code and submitting it to projects that are completely overwhelmed
No, this is a result of unintended consequences.
We made "Github contributions" a metric for people applying for dev jobs. So, of course, because devs are the kind of people we are, they started working out how to game that metric.
Some folks decided to start paying bounties on bug fixes, features, etc. Those bounties are fairly trivial by western standards, but are significant for developing countries. This creates a new career for developers; racing to collect the bounties on offer.
LLMs have exacerbated these problems by allowing existing people doing this to do it faster, and also allowing more people to pretend to be software developers and get in on the action.
If we stopped allowing LLM-authored contributions we'd still have too many shitty PRs. It would just be back to pre-LLM levels of "too many".
The answer is to make Github contributions valueless. Stop paying bounties, and stop using them to assess candidates.
> and for some reason tons of independent developers
Cowboy coders got a virtual cowgirl coder and sold it to everyone, hmm, maybe... (respected or not, solo devs don't always have the requisite skills to not be a cowboy, either due to lack of experience or lack of innate skill)
I don't know that I completely buy this narrative, though. There has been a strong, top-down push for this since the "beginning".
The astroturfing successfully broke a lot of people's brains
Makes me wonder if an ELO-based system would work to mitigate these issues. People who merged PR successfully onto a project, that had real issues acknowledged, the quality of their responses measured by other users reactions or something, etc, multiplied possibly by the degree of importance of the project where their activity has been made. Won't be about human vs AI, but actual helpful effective being vs low effort/spammy contributions. Issues and PRs could be sorted and filtered by their ELO score. I'm saying ELO as analogy to "score based given the context", not really a 1:1 translation of the ELO system.
Negative score would be reports from other users because of spammy content or not acknowledged issues, with a middle ground of neutral score (+-0) or little positive score to issues or whatever with clear good intention, but couldn't reach a proper merged PR or were not issues (e.g. issue existed but wasn't the correct repo to be addressed, PR was good but needed other stuff to be implemented prior to it, maybe in the long run, etc)
ELO is shockingly easy to manipulate. For example there was a literal jail with a decent chess player in it. He created a pool of players who got great ELOs by beating him, then used them to boost his rating higher. Wash, rinse, and repeat.
Given any manipulatable scheme, AI will figure out how to manipulate it. For the OP, what happens if a single AI manages to get through to contributor? Then it starts elevating other AIs to contributor, and we're off again. There doesn't have to be a purpose to this. Trolls will troll, and trolls armed with AI bots can devote endless energy to doing so. The more you work to keep them out, the more fun it becomes for them.
I wish I had an answer for that problem. But I don't.
ELO is a bad fit because it requires competition between submitters; but if the idea is interpreted as “contributor karma score” or similar (not everyone’s familiar with the mathematical nature of ELO), then the way to close the loophole is to only consider voting inputs from the human project owner. This project chose to have people lie to a webform rather than lie to a git interface about using AI, so I don’t expect it will be particularly successful at inhibiting AI use by project-involved humans, but certainly it’ll squelch a lot of noise from unattended/passersby.
I think they were saying Elo system as kind of a general ranking system idea instead of the actual algorithm.
You could probably use some kind of pairwise ranking algorithm (like anything based on the Bradley-Terry model) to rate human vs. AI contributions, but that would take a lot of manual effort. Google is using it to (supposedly) improve their searching algorithms. They give testers two different versions and ask them what's better.
fix this problem by make the rating value tied to some paid currency - a repo owner would have to pay for the PR, and that PR contributor will now have more currency than previously. In order to have said currency to pay, the repo owner would need to have contributed to another repo whose owner have currency.
The totality of someone's currency is their reputation.
Of course, now the decision becomes...who is the central currency issuer that creates it?
It's the StackExchange model! This has bootstrapping issues, is hard to break into the community, and risks creating moderator cliques.
This is called proof of stake
>what happens if a single AI manages to get through to contributor
Then they'll get removed by the humans? Its about cutting down work, not about eliminating the work entirely
The current approach removes about 99% of their overhead it would seem. If they have to do a few manual interventions here and there, that seems like a huge win overall
Reputation scores, review cartels. This all sounds familiar!
contributors being able to grant contributor to other users seems like a problem
For those wondering what Elo means, it is a person's last name, not an acronym (not all caps). More info here:
https://en.wikipedia.org/wiki/Elo_rating_system
Thank you, big fan of ELO https://en.wikipedia.org/wiki/Electric_Light_Orchestra and I was a bit confused about the comments.
It's Elo not ELO. Elo is not an acronym.
https://en.wikipedia.org/wiki/Elo_rating_system
That’s a fun fact!
From what I've seen in the comments, it's definitely ELO, if not through ubiquity alone. Happens to the best of 'em!
Elo is nicer as it gives a nod to the inventor, no?
Some say the greatest compliment a mathematician can be paid is when a concept that bears their name is so ubiquitous that we no longer capitalise it. (E.g. Abel with the abelian group).
I have built something like this and in process of collecting the data.
Frontier users: 527,865 Light indexed: 527,865 Ready to queue: 9,083 Fast scores ready: 0 Activity events 24h: 30,266 Fast scores completed 24h: 19,123 Deep jobs completed 24h: 3,043 Fast-score ETA: n/a Deep-hydrate ETA: 69h Stale running jobs: 0 GitHub backpressure jobs: 19,113 High automation signals: 4,608 Medium automation signals: 1,327 Completed jobs: 74,714
Biggest challenge is Github's rate limits. At this pace it will take two more months to have 98% coverage. But after that the maintenance should be quite straight forward.
Sounds a bit like Mitchell Hashimoto’s Vouch: https://github.com/mitchellh/vouch
This would just hurt new users similar to how you are unable to comment on 90% of subreddits on Reddit as a new user, because you don't have enough karma points, or how on Stackoverflow your permissions are severely limited until you do certain jobs. The incentives aren't very good in systems like this. Bots can be made to easily game the system while regular users are discouraged from even participating.
Some kind of vouching or scoring might make sense to help qualify contributions and many people have suggested similar recently. If by "ELO-based system" you meant "some kind of scoring system (not based on Elo)".
The Elo rating system doesn't make sense in this context; it's designed around collecting zero sum game results for a given community of players and building a model around it.
I think you need trust circles, not ELO.
The problem is you want the ELO score based on work on other community projects - you can't assume good faith here.
The problem with that is that there are certain kinds of users that like to take control of community projects. And then they take control of more, and bigger ones.
There are a lot of political tricks that get used.
What is scary is that one of those kinds of users are malicious state actors. Like North Korea and Russia...
> It's not a contract job— it's our optional way of saying thank you to the community.
The writing style in their onboarding doc has common AI tells (in the quote: em dashes, “it’s not A, it’s B” sentence).
I can understand that, perhaps they want to fight fire with fire or don’t have time as they already say. Still, it all feels like inadequate half measures to me.
The entire post is clearly LLM generated. I get that a person clearly put together some thoughts, but prompting an LLM to 'turn this into a blog post' is the kind of low effort content I thought was not appropriate for HN.
At least bringing up the underlying method (restrict to contributors) has spawned the discussion about how that's probably a bad idea on the security side.
Using AI for your own project is different than being overwhelmed by AI contributions from other people/bots
I hope he is going to find the seasoned engineers that he is looking for
Is the solution to everything simply more catgirls [1]? Proof-of-work was, after all, about countering email spam. PR spam is but the latest in that long tradition.
1- https://anubis.techaro.lol
Proof of work doesn’t work here same as it doesn’t work for email. The effort to mint a valid PoW is always going to put the legitimate user at a disadvantage, whatever the implementation is. Someone with an incentive to spam will always be able to do it faster, more efficiently than you.
You can’t submit a PR because your laptop is too slow? Rent some hash rate from someone, and now you’ve just made a system of paying botnet owners to be able to make a typo fix on a github repo. HashCash was never used in the real world for a reason, it sounds cute but the incentives are so insane as to only work in a vacuum where you assume everyone isn’t cheating.
> Someone with an incentive to spam will always be able to do it faster, more efficiently than you.
Sure, but looking at the cost to do it at scale is the wrong metric. I surely can't compete with a career spammer on emails-per-second or even emails-per-dollar, but I also don't need to.
It's more about the expected-value versus the cost. For example, my expected benefit from one email to my family is (while hard to quantify) hopefully much higher than a spammer's expected benefit of one spam email going out, which has a very small chance of leading to any amount of money. Attaching a CPU-churn cost per email is something I can ignore on my desktop, but they have to at least budget for it.
I'd also like to note that the win-condition isn't as extreme as making spam (or other "crimes") truly unprofitable, it just needs to be less profitable than other things the time/resources could be used for.
Agreed, PoW is an especially poor solution here.
We really need to solve SPAM itself here, I think there may be a way to do it. I.e., the problem of spam is NtoN scaling connections. The network has never been able to solve that problem (exponential is the hardest). Limiting communication in terms of mesh networking may be the ultimate solution - bots can't get to you because they can't reach you.
What needs to be invented is a bridging protocol - some way to establish "legitimate" lines of communication over a network, while preserving (to some degree) privacy and decentralization. AI can only enter this network by being explicitly added to the channel, and thereby explicitly and easily blocked (and also solving the general SPAM issue once and for all).
Just like we did with IP addresses. If yours is blocked by Cloudflare, you can pay a botnet operator a few dollars to use theirs! You can even use your credit card through a mostly-legitimate website. It's very convenient.
Anubis is actually not a cat. The original Egyptian deity is a god of death, and has a canine head. Anime catgirls and dog girls can look similar at first glance.
I believe they are referencing the person who wrote the program, not the name itself.
I think Anubis is against crawlers, not against agents that make PRs. PoW doesn't work here, the agent will just do the computation.
The irony of the .ai domain.
I dont think anything is ironic about it because they aren't suggesting AI is bad. Just that it can be misused.
Thanks for pointing it out. It has eluded me and it's incredibly funny
Not just the domain: it's an agentic stack! In other words, I could use their product to create the exact type of PRs they're lamenting here.
also, could the website plz fix its scrolling code? its annoying. i can't read the article
Would love to! Could you please share more? I can't quite see the issue
"I never thought AI would slop my project!" Says company centered around AI slop
> Should we stop giving fun test tasks to our job candidates?
Yes
It seems this particular company makes a payment for completing those tasks, so it might not be that bad.
We do, it's a part of our hiring pipeline: https://archestra.ai/careers
Developers: stop doing whiteboard interviews, they don't measure anything relevant to the real job
Also devs: stop giving us real world problems to solve
Those are the only two options to finding quality candidates?
Try talking more about the meta of coding itself. Get into the developers head by _talking_ to them and understanding how they would approach and attack different problems. You can show them code and ask them what they would do differently / how they would go about implementing X-Y-Z. Just because you can write foobar doesn't mean you understand how to apply algorithms or w/e specific problems [your] team has. It's _far_ better to understand how they would solve a problem over their syntax anyway.
Yeah, fun for who exactly?
Me. That sounds way more fun than inverting a binary tree, and they pay candidates for their time.
> If the email matches their GitHub account, GitHub links the commit to their profile and grants them contributor status.
When the article mentioned email matching, I was concerned that it would break down when a contributor's email address changes. (I have contributed to more than a few projects over the years, using email addresses that no longer exist.)
However, it looks like they're not using the email address recorded in the author's original git commit, but instead a GitHub-generated address whose unique parts are the GitHub user ID and username. That should survive authors changing their email addresses. It would still break down if a contributor loses access to their account and has to create a new one, but that's probably less common.
While git has always allowed this, I don't really like the idea that someone can write some code, slap my name on it, and push it to their repo.
You can sign your commits with OpenPGP.
I think this is why signed commits are also supported. My first thought was that this probably doesn’t work with signed commits. But, maybe it does since they are listed as the commiter.
Yup. At the very least, the "big dogs" aka Github and Gitlab should allow you to "claim" an email address to an account and only link it up when the commit in question either directly got authored from the web UI or got cryptographically signed.
Hi HN community, I wanted to share our approach to reduce amount of AI slop PR's and issues in our repo. We enabled "require prior contribution" flag on GH and created a CI script that creates a tiny commit co-authored with you, if you pass captcha on our website. Worked really well and we were able to block at least 500 bots in the first week. Sharing a screenshot from cloudflare: https://archestra.ai/hn-comment-cloudflare-challenge-outcome...
Yep, this is similar to some other version control tools like Tangled which has vouching.
https://blog.tangled.org/vouching/
Who do you add as a contributor though? Wannabe-contributors? Then they appear in the list of contributors before you even see if they're capable of producing an acceptable PR.
Your solution would be great if GitHub would also allow me to whitelist specific users, but unfortunately this still won't block "implementation plans" in comments.
Thats a really elegant solution.
How does the website trigger the CI script? Through GH rest API?
thank you, yep through the rest API, here is the example: https://github.com/archestra-ai/website/blob/29ebdacbd8a22b9...
You should release this as a service.
Woudln't it be trivial to farm the stats needed to pass the bot checker's theshold?
I'm not sure why gh hasn't already implemented stricter measures / filters / tools for PRs. It would cut down on spam and also help save their servers that can't handle the increased AI load!
Repos get forked, code gets pushed, all before a PR is created. What kind of measures can be implemented to cut down on the AI-general forks and pushes?
You can fork and push all you want. The problem is specifically when you show up in my notifications with your junk PR.
The issue for GH isn’t your PR spam. It’s all the other operations before your PR spam ever arrives.
Sounds kind of weird that the blog post complains about `poisoning the conversation with pointless "implementation plans"` when literally they ask for that, after attaching $900 USD bounty to a very under-specified issue, and even replies with "Do you have an implementation plan in mind?" to some of the first "attempters". Sounds like they got exactly what they'd been asking for, and even before LLMs if you pulled something similar, the effects would have been similar.
It's fine for developers to provide a plan, even if it gets rejected. The problem comes when every script kiddie figures AI has made them into a developer.
Imagine you want to get a doctor's opinion, or maybe a couple of opinions. But a zillion AI-amateurs have registered themselves as doctors. How do you separate wheat from the chaff?
> Imagine you want to get a doctor's opinion, or maybe a couple of opinions. But a zillion AI-amateurs have registered themselves as doctors. How do you separate wheat from the chaff?
Right, but that's not what happened though.
Someone went to the public square, said "Hey, I'm looking for any sort of doctor, and I'll pay you $900 if you tell me your plan and then whatever plan I chose wins" and then they get surprised they get flooded by zillion AI-amateurs.
You don't generate a ton of chaff then try to find the wheat, you ensure your process doesn't generate a ton of chaff in the first place. Offering large monetary rewards for relatively simple work for anyone in the public is bound to generate a ton of chaff...
This is great example of the toxic effect money has on open source. Reward people with respect and recognition instead. Weird anonymous accounts no one's ever heard of will leave, because someone (or something) who's concealing their identity has nothing to gain from recognition. Honestly GitHub should have a real names policy. Because if you're not Satoshi Nakamoto then there's only three reasons I can think of to be anonymous on GitHub: (1) to avoid obtaining your employer's authorization, (2) to spam, harass, and engage in toxic behaviors, or (3) you're not even human. All three of these are the last things I want when engaging on the GitHub platform. Don't get me wrong, I love robots. But I'm perfectly capable of talking to the robot on my own. I don't want to talk to your robot. I also don't want people slipping me intellectual property below the board without their employer's consent. And I certainly don't enjoy all the hate and harassment. GitHub has tried to help with the last part, by making overt displays of hate something that can get you in trouble. The issue is that people just get more guilesome with more anonymous accounts, because the issue was never disrespect (which can actually be strategic and pro-social if we look at Torvalds' career), but rather bad faith participation. If GitHub can guarantee that all its users are human real names good faith actors, then we might be able to start talking about open bounties.
> someone (or something) who's concealing their identity has nothing to gain from recognition
The xz supply chain attacker hid their real identity, created fakes one and gained recognition over time in order to gain more access and add the backdoor. So TLAs and other bad actors at least are interested in gaining recognition.
I know, right? It's like, finally—a threat actor who's intelligent enough to understand what capital means in the open source community and is willing to devote resources to engage with it authentically (even if it's for evil nefarious ends). The xz incident showed that the open source community has many other good defense mechanisms for verifying and spotting malicious work and then solving it. But we won't even get to play that game if we're inundated with anonymous agent spam so that GitHub can juice its MAU numbers. Maybe they should require every account buy a $40 yubikey. I don't know what the answer is. But I know that no one gains when your measure of success is driving the cost of burning open source developers out down to literally zero.
My first thought after reading the blog was, let me share the blog with Claude and ask it how bots can circumvent this.
imo AI bots have significantly affected OSS and we need better qualitative measures to define success
I like how they are taking a stand against vanity metrics. Rare to see that these days
How is the status revoked without rewriting git history?
we can block the user in github ui
'I will take "problems that could be easily be solved by implementing a Pfand system" for $200, Alex.'
Seriously. Just ask for a US$10 deposit for the each PR. If the PR is accepted (not even merged, just accepted as "this is a good effort"), give it back. Hell, give double the amount for good effort and you got yourself a cheap way to attract good contributors.
Best case, bots will balk at the payment. Worst case, the funds can be used to hire someone specifically for triage.
This sounds like a great idea until you think about it for more than 30 seconds. Similar to most "it's so easy, you just" ideas.
Seriously, chill, then think about how you'd implement it. Then think how it'd go wrong. Then think about how to fix those problems. Repeat until you realize there's a better solution or until you solve the problem without making it overly convoluted. More often than not the former is the better option. More often the latter is just a variant of the sunk cost fallacy and your ego. Reality is (un)surprisingly complex and solutions aren't usually trivial
This is an overly negative response to a genuine solution. There are a million reasons you shouldn't do X or Y.
More than likely GitHub would have to maintain their own internal wallet solution for this, which is a big engineering lift. But we're all just having a discussion.
There's two types of lazy, and this is the kind that creates more work, not less
Then people from a sanctioned countries are blocked from open source, or worse, you have to explain to the bank and/or the government why you sent 20USD to someone in Venezuela.
And here we have another specimen of "things which crypto are actually useful" spotted in the wild!
I think the intersection of the set of people able and interested in contributing and those who are willing to figure out cryptocurrencies is the empty set.
I quick visit to https://gitcoin.co/campaigns will show you that you are wrong. Hundreds of projects funded by even more people.
Mind you: that's on one of the most convulated ways there is to get involved, because it involves a bunch of smart contract operations and on-chain voting. If we are talking about crypto only as a payment network, things are even simpler.
It's cheating if the projects using it are cryptocurrency related :)
A generic python library used by generic people who have no interest in this field is something else.
Coinbase reports more than 100+ user accounts worldwide. Kraken has ~10M.
Also, we are talking about people who are tech-savvy enough to be interested in participating in a FOSS project. Opening an account at an exchange is not rocket science.
No, getting the money out of an exchange if you are in a country USA doesn't like is rocket science. Which was the whole point of using crypto rather than money.
The exchanges are for the people that can send the money, not receive it.
And those who are living in countries the USA doesn't like will probably have no issue to learn how to work with crypto. Of all the complex things they need to do to work around the restrictions, setting up a wallet barely registers.
This is exactly the strategy that the owner of the SomethingAwful forum used in 2004 to get rid of bots and assholes. (I used to remember his name, kinda famous, oh well).
https://en.wikipedia.org/wiki/Richard_Kyanka
Lowtax, I think is what you're looking for.
Possibly the worst idea I've heard this month.
No one, meat or chip, would just set aside $10 "for the opportunity to contribute"
This is "let them eat cake" level of out of touchness.
I have 3 PRs on https://github.com/django-oauth/django-oauth-toolkit/pulls that haven't been merged for OVER AN YEAR due to the maintaners being overloaded and who are expected to work on this for free. The fact that these PRs are not being promptly reviewed have cost me at least 3000€ in potential grant work.
If I was told that I could make a deposit of $10 to get less stressed maintainers and a faster PR review cycle, I wouldn't even blink. I wouldn't even ask for the money back.
How did it cost you money in grant work? Can't you just fork and use that?
This is an evergreen internet comment right here. Condescendingly proclaiming "This problem could be easily solved by [significantly worse solution that had 1/10th the thought put into it as the actual solution by people with a stake in actually solving the problem rather than making quippy armchair comments]".
---
I know it's against convention to comment on downvotes, but really? Really? This is controversial? The OP came up with an elegant solution that cleanly solved their problem without subjecting contributors to anything more than a captcha. Then somebody comes along and says "oh, it's so easy, just charge $10". You're going to set up payment infrastructure, incur administrative overhead with human support managing refunds, and deter 99% of actual humans from contributing, and then call that the easy solution that OP is so stupid for not thinking of first? Give me a fucking break. This site really is just Reddit-lite, anyone who thinks about engineering problems seriously would realise this does not stand up as anything beyond a pithy internet solution with three seconds of thought into what actually implementing it would entail.
Github already has the payment infrastructure.
Polar.sh is already doing things that are a lot more complex in this space.
If you are in a civilized country which allow direct payments (i.e, anything but North American nowadays) and you don't want to deal with Github or any external system, there is always good old "make a M-PESA/SEPA/Pix/UPI transfer to account XYZ")
> the thought put into it as the actual solution by people with a stake in actually solving the problem
Let me flip your argument: think of how much time and thought is poured into problems like this one by people who don't even try to implement a Pfand system beforehand.
> Github already has the payment infrastructure.
...which is not available to maintainers to use in this way.
> there is always good old "make a M-PESA/SEPA/Pix/UPI transfer to account XYZ"
And then lock out anyone who is not from the same country as the maintainer, on a platform that is known for its global reach.
Moreover, you're introducing significant anti-human friction. For privacy-conscious people, it's a complete non-starter; I'm not giving my payment information, not for a $1 transaction, and compromising my anonymity just to make a PR for the benefit of other people. That's a small subset. Then, you have the lazy people. The majority of the population will simply not bother with something if it has friction. Getting out their credit card is one of those things, and it's why products/services that offer free trials or a free tier tend to be overwhelmingly more successful -- people want to see a tangible benefit to themselves before they engage in high-friction processes (where "high-friction" is as little friction as requiring a payment, yes). "Free to play" video games with microtransactions engineer first-time purchases to be cheap ($1 or $5) and have 5x or 10x the value of the normal microtransactions, because that first hurdle of getting somebody to hand over their payment information is by far the biggest.
I'll take the captcha, thanks. And maintainers will too, because they'd rather have the solution that filters bots and keeps humans contributing rather than the one that filters out both humans and bots.
> significant anti-human friction
Yes, that friction is intentional. The lazy people don't want to do it? Great, there is very little chance their contributions are worthwhile. The privacy conscious people won't do it? Then let them work on their own repositories and complain loudly about the idiot maintainer who puts these insane barriers. Then the maintainer can go take a look at that forks done by the loud complainers and see if it is worth to whitelist them.
> it's why products/services that offer free trials or a free tier tend to be overwhelmingly more successful
Drug dealers also offer the first hit for free, why don't you use that as an example as well? ;)
To answer this properly in case the quip was too vague: there is no reason for "number of PRs opened by new contributors" to be a viable/interesting KPI for any FOSS project.
> I'll take the captcha, thanks.
First you need to show me all your cool FOSS projects.
How is it "laziness" to not want to pay $10 to submit a bug fix to your repository?
1) I just used the term lazy because that's what OP use.
2) You are not "paying" $10. The money would be returned to you. In case you haven't heard of Pfand systems: https://en.wikipedia.org/wiki/Container-deposit_legislation
Yeah, because we'd hate to allow people from poor countries to contribute to FOSS projects, right?
Or teenagers without full access to online banking.
Or the unemployed.
Oh, give me a break. No one is taking the ability from others to fork the repo. If these exceptional cases really were to happen, how fast would it be for someone else to notice and do one of (a) notify the maintainers to get this particular user whitelisted or (b) front the entry costs?
Sounds like bandaids on top of bandaids, at which point you start to wonder if the idea is fundamentally broken.
> Sounds like bandaids on top of bandaids
And a registration system that amounts to a more complicated captcha doesn't? How long until someone starts farming accounts and run bots that jump through these hoops as well?
> wonder if the idea is fundamentally broken.
It's only "fundamentally broken" if you need to build a perfect system that needs to accept 100% of legit PRs without raising any level of friction.
But we don't need that. Pfand systems are not meant to be perfect, and they are not meant to the single solution to any problem involving the commons. They will not get rid of all bad behavior, but they will certainly bring it a global-scale problem down to levels that can be then managed by other smaller, context-aware systems.
I semi-regularly offer drive-by PRs to projects I like and use. They're real PRs, not generated with AI. They range from papercuts to doc fixes to attempts to add that one feature that I want. Sometimes it's a drive-by group of PRs. Or an issue and a PR. I try to conform to what the maintainers prefer.
Unless I knew the maintainers personally, this would prevent most of my contributions, which are most often accepted. Maybe it's worth losing out on my small contributions to avoid slop. But things would absolutely be lost this way.
Agreed... However there's not a good IsThisAnAI() test at present. So unfortunately, we will have to use anti-spammer techniques (because that is exactly what AI is, high(er) quality spam).
Why should I trust you to give me my deposit back?
Because the cost in reputation is not high enough to justify a large scale scam operation?
If you don't trust the maintainer, you can always fork a repo and let them merge on their own.
Interesting idea, I wonder about using it myself.
Let's say I'm a maintainer of an open source project on Github/Gitlab. How would you actually implement this deposit-refund loop in practice?
I believe you are asking me in jest, but if you are genuine, this is what I would add to my CONTRIBUTING.md
``` # FIrst-time contributors
Due to the increased number of AI bots and low-effort contributions, we are being forced to add some friction for first-time contributors. PRs are closed for anyone not explicitly added to our list of authorized users.
To be accepted in the list, you must do one of the following:
- Show a history of meaningful contributions in projects from related technologies done before Jan 1st, 2023.
- Be vouched by one of the existing contributors in the core team
- (If you have github sponsors/polar/patreon) Be a sponsor for the project for the last 3 months)
- Submit a small payment, which will be held in escrow until your PR is accepted. The following methods are accepted (choose all that apply: paypal, SEPA, Crypto, Venmo, Pix, UPI, M-Pesa, etc) ```
Oh no, I'm being genuine. Documenting the process itself wasn't what I was curious about, I was wondering how you'd go about your last bullet. Accepting lots of currencies can be hard, but I guess I'm not super familiar with online escrow services. I'm not sure how simple they can make that process, or who would pay the cost of using them (I assume they're not free).
I was also wondering how automated or manual you would envision the review process. I'm guessing your hope would be that the small deposit would stem the flow of submissions enough to make it all possible to review manually again, and you would also manually return all the payments sent to escrow?
Yes, I'm assuming that adding requirement for payment would bring the number of requests down to a level which I could manage with a simple spreadsheet.
Paypal/SEPA transfers are free in Europe. And even if I lived in the US and had to pay a small processor fee, I'd be more than willing to cover the $0.50 in fees if that meant I was receiving contributions from people who went through all the trouble.
$10 to a Silicon Valley software engineer reading this comment may feel like... $(a lot?)... to a range of other would-be contributors (thinking of $6/day minimum wages in some places, for example)
Wonder if a dollar would work for now until more people give bots credit cards.
Easily? You think the kind of people who think it makes sense to make bogus slop PRs are going to react reasonably to overburdened volunteer maintainers refusing to give them their US$10 back?
Yes. Once a PR is rejected, contact from that bot is blocked. No appeals.
This is never going to work. Sufficiently many of these people are going to find maintainers' home addresses and send them death threats and the likes. If you see how badly some people flip out just because their PR is rejected, it's going to be much much worse if their PR is rejected and their money is taken.
Ok. If I'm the maintainer receiving death threats over that, I'd tell them they would get the $10 dollars back, plus some extra money for their troubles.
Location of the envelope with the money: the same police station where I'd reported the death threats.
The worst case is that someone loses out on $10, no? How does this work if the maintainer is the swindler?
I don't think that is a (very realistic) concern. AI is slop, the problem is not that the real contributors are struggling to get PRs merged.
The bigger issue being, raising the bar to students who may have otherwise had productive careers (but education is a general issue, where the students don't even yet recognize they are being scammed).
I don't follow, and I'd be concerned that this opens up a cottage industry of bots generating plausible looking repositories that unwitting contributors would attempt to contribute to. We already know that bots are astroturfing repos to generate overinflated star counts. I'd say the least crap option here is to honeypot PR contributions from bots
This feels like bot logic, lol.
Unless the contributors don't care about the repos they contribute to, this is not a likely scenario. AI doesn't care. We do.
What is bot logic exactly?
You keep describing this as not a likely or realistic scenario. But why is the likelihood even of relevance here? The way to avoid the worst case i.e scammed of your money, is to not even put it on the table in the first place.
> What is bot logic exactly?
Ill thought out logic like your own. I think you are likely a bot at this point.
It's not likely, because that's not something that people are likely to do. Only a bot like yourself with a poor model of the world will do this type of thing. It will be amusing to see the AI bots trying to run the scam you are describing and then nobody will contribute to the fake projects... except other fake AI contributors.
Dude, you're claiming that there's no likelihood of people getting swindled out of their money by handing it over to strangers. So your reaction is to play the bot card? We're done. You're clearly not arguing in good faith here.
> people getting swindled out of their money by handing it over to strangers.
I think that OP is trying to say is that there is very little reason for a human to go through the trouble of contributing to a "plausible looking fake repo".
To get to the point that a repo starts to attract interest from other contributors, that project needs to have actual utility.
Who in their right mind would jump into opening a PR from projects they never used? And if the project does get used to the point that it attracts people interested in contribute to improve it, wouldn't it mean that we've achieved https://xkcd.com/810 ?
So I pay $10 when your bot fucks up?
That's called theft. And for what, one banana?
Obviously, the triage should be done by a human and not automated.
Doesn't that put us into the same position?
Let's also be realistic, everything that can be automated will. Even if that thing is worse off for it. There's a clear historic pattern of this. Companies and people love to be penny wise and pound foolish.
> Doesn't that put us into the same position?
Of course not, because the number of low-quality PRs with $10 attached to it will be lower than whatever number of PRs are being created now.
You also lose out on a lot of would-be PRs. By people who don't have the money, don't have trust, or have a visceral "fuck you" stance. There's a lot more reasons that this suggestion creates a gate that dissuades the people you want. I stand by that the solution is naïve, but you're welcome to give it a try on your projects. I'm sure it'll be effective at reducing a lot of spammers, but I'm also pretty convinced it'll come with a large false positive rate, which is invisible (giving you false confidence)
> You also lose out on a lot of would-be PRs.
I am more concerned about the sustainability of the projects as a whole than trying to optimize the number of potential random PRs.
> you're welcome to give it a try on your projects.
It's not exactly the same, but in a way I'm already doing that with Communick. I'm running one of the few Matrix and Fediverse services where members must pay to have access. Up until last week, I was giving 14 days as a free trial period and no deposit/confirmed subcription required. But now because of AI bots, I dropped it and I am collecting payment info before activating any account.
If I were playing the startup playbook, that would be insane. It's already crazy to try to charge something that people are used to get for free; my conversion rates are already low, requiring credit card info will make them even lower.
In the end of the day, I don't care. At first I was really hoping this would be something profitable, right now I just keep it running because I can. Even with the small number of users Ithe servers get, I get enough to cover the better part of operational costs, I get to sharpen my devops skills and have a test lab to learn what is that people really want (i.e, they are willing to pay to have solved) vs what people claim to be a problem.
All in all, the Communick instances are not going to win any popularity contest, but my servers have been up and running without major issues or drama for more than 6 years, and that's a lot more than I can say for all the other servers that have come and gone because the admins tried to play the numbers game.
You can't just hand wave them away as if this isn't an important factor. If you don't care about them at all I got a much much simpler solution: don't allow issues or PRs. Problem solved! But that's not a real solution either
> You can't just hand wave them away as if this isn't an important factor..
There are plenty of ways to indicate in the project that the Pfand is meant as one way to filter out bad actors, but it doesn't mean that it should be the only way to accept external contributions. You can find somewhere else on the thread where I listed some alternatives that can be used as well.
> If you don't care about them at all I got a much much simpler solution: don't allow issues or PRs.
Yes, and what is the problem with this solution? That's what many projects are doing and many more will do. They will close access to non-members and only accept someone new when they have some type of social proof. [0] And that is totally fine.
[0]: https://news.ycombinator.com/item?id=43423063
If I have to trust you to give me back my $10, I'm never contributing to your code. Ever.
If I have to trust GitHub to give me back my $10, frankly, I have more trust for a random person on the internet at this point.
Also, you glossed over my banana joke, but it did hold meaning[0]
Are you serious? I mean it is an acceptable solution but it's completely orthogonal to the one we've been discussing. I can see you're not serious. I was skeptical because the first comment, but thanks for making that clear now.[0] https://www.youtube.com/watch?v=Nl_Qyk9DSUw
What? No. A PR is me giving my time to the project. I don't get anything out of it except the warm feeling of having helped out. If I have to pay money to submit a PR then I'm going to play video games instead.
> A PR is me giving my time to the project
Unfortunately, the issue is that time is not enough of a filter anymore. The time from machines is basically worthless compared to yours, so you need to give something else, and that something else needs to be something that shows you have actual skin in the game.
> so you need to give something else
Well no, they don't need to. As they said they could just do something else instead of contributing (and I know I would too).
Your proposal would just end up killing those open source projects even more than what you are trying to solve.
> Your proposal would just end up killing those open source projects
Name 3 open source projects that are dependent on a continuous stream of first-time contributors to keep going.
AI company annoyed by AI ... Surprise
I don't have a better solution, unfortunately, but it doesn't seem seem to like the spam problem has been solved. It has just been moved from pull requests to commits:
Currently, more than 10% of all commits in the archestra repo are essentially noise (369 of 3521 commits), accounting for more than half of all commits in the last month (303 of 578 commits).
But maybe (probably) the amount of such commits will go down over time, compared to the growing amounts of AI slop
As those commits were made from our system they don't create any noise for us, as PR/issues/email notification do. We only include real people who could solve the captcha and their input is mostly valuable
Signed Commits from known authors would also help!
so...they are manually re-setting the "interaction limits" over and over again, since they are only temporary?
why not use hooks to automatically reject issue comments / PRs etc. from users that didnt go through onboarding, rather than repurposing GH features that aren't really designed for that use (and are hence in danger of being changed someday)?
GH sends the email notification to all subscribers at the moment of posting a comment. There is no cooldown or a way to unsend the notification using hooks
submitting attempts — but soon...
not just this issue — but the entire repo.
contributors like @ethanwater, @developerfred, and @Geetk172 — people actively working on bounties — were getting buried.
two identity fields — author and committer — and they can be different people.
metric growth — a substantial part of
> Final Words
> While GitHub reports massive metric growth — a substantial part of which is AI-generated — we as an open source project team have to do the heavy lifting of cleaning up AI slop from our repository and come up with esoteric workarounds to keep the level of legitimacy of our open source audience.
AI generated slop!
There's got to be a concept to differentiate the industry plants who start an "open source" project that has enough funding for a $900 bug bounty. They are speaking and developing in the language of corruption and they don't even know it. Of course you will receive AI bot spam, but unfortunately it will continue if you don't take a hard look in the mirror.
I don't understand how clicking "I agree" a few times will stop the AI bots?
The captcha - maybe.
Why does this company use the Slashdot logo?
It's so ironic as the website screams vibe-coded design on top of that.
reindeer games
It's quite ironic to complain about AI slop in a piece that's quite clearly AI slop.
Soon there will be no more AI doomer comments. The bots will take over that job too.
---
I'm working for an open source company, and my God, are 95% of contributions useless.
There are really dumb ones where the bot writes 10 paragraphs about how he implemented the feature, but the entire changeset is adding one line to .gitignore or adding a CLAUDE.md file.
There are even worse ones where the bot submits 3000 lines of code that seemingly works, but you have to spend an hour to figure out why it doesn't work.
The dumb ones are so much better.
See, this is an article that uses dashes correctly. It adds value, creates a bit of buildup
This is funny to me because the title on this submission currently refers to “Git's –author flag”, which is an extremely incorrect use of a dash. (The original article doesn’t make the mistake. Not sure if the error is from the submitter or from an HN title mangulation.)
cool
Interesting
Musk before the verdict: "It's not okay to steal a charity"
Altman after the verdict: "It's okay to steal a charity"
That's a neat way to interface with GitHub's authentication system, but I don't see how they've solved the fundamental problem because their whitelisting process is just "click ok fine 10 times". Why won't the slop peddlers just do that too?
I think the point is to add a bit more friction to the process. You want to make it so that people can do it with minimal effort and an AI bot will give up. If you're in an arms race over AI commits and PRs, this is a decent middle ground to start from.
(Why there is a race for AI commits/PRs to projects is beyond me though...)
click ok fine 10 times + captcha seems to be working fine
For now…
Until the AI learns the workflow on the next model update, indeed.
What I see is a (clever) hack, and GitHub continuing to provide good tools to its users.
What I see is a solution for a problem that is self inflicted, meaning lumping contributors and generic internet users in the same workflow. In big projects, you have the core team, a handful of well known contributors, and everyone else.
I strongly prefer the git email model, where it’s often trivial to control the flow of changes proposal. GitHub does not have the same wealth of tools and versatility.
I seem to have completely missed the "failing" when writing "this is GitHub failing to deliver tools". My bad