The system is new (built last week), so I don't have reliable production data on false positive rate yet.
Right now my current detection logic is basically:
- 25%+ CCU growth vs 7-day baseline
- YouTube video mentioning game within 48h
- Keyword match (currently using regex and exploring other methods)
If I have to guess where the false positives are going to come from:
- YouTuber plays game (drives CCU) but doesn't mention new mechanic
- CCU (concurrent users) spike from external event (streamer or holidays)
- Generic update videos that don't indicate mechanic type
Next step is running it for 30 days and tracking precision/recall. More of that in the GitHub itself.
Appreciate the question, since it's the main thing I need to validate.
Looks like you've committed `.env` file with private info. I suggest you change token immediately and add `.gitgnore`! That's a *security risk*!
It's best practice to keep env files private, but commit something like `template.env` with placeholder values.
Other than that - looks like a must-use tool for every roblox studio!
Thanks for catching this. Strange part is I actually had added this to my git ignore last time, so maybe something went wrong in the process.
Highly appreciated!
Thank you for the feedback as well :)
How many false positives?
Ty for the question.
The system is new (built last week), so I don't have reliable production data on false positive rate yet.
Right now my current detection logic is basically: - 25%+ CCU growth vs 7-day baseline - YouTube video mentioning game within 48h - Keyword match (currently using regex and exploring other methods)
If I have to guess where the false positives are going to come from: - YouTuber plays game (drives CCU) but doesn't mention new mechanic - CCU (concurrent users) spike from external event (streamer or holidays) - Generic update videos that don't indicate mechanic type
Next step is running it for 30 days and tracking precision/recall. More of that in the GitHub itself.
Appreciate the question, since it's the main thing I need to validate.