I like Pokemon Go and play almost every day. I did this scan one time and then stopped. The rewards are not worth the hassle. I don't think many players are doing it. It's just very weird to stand somewhere and scan an object.
I also wouldn't say 'unknowingly trained', it's pretty obvious what it does, and I think the game even tells you that they want to understands how the POI looks like in 3D.
30B images isn't that much in the context of Pokemon Go playerbase of ~50 million (conservative estimate based on users today). That's about 600 images per person, in a game that has been out since 2016... that's pretty low adoption as the previous user said. I don't think the quest has been out since 2016, but considering a large fraction of users are basically daily users, it's still quite a small number of images.
To maintain that take, wouldn't you need to offer a plausible way that Niantic managed to train their Visual Positioning System using that data if the data was all bad?
I guess we don't know the terms of the deal, as far as how much they paid? So maybe they didn't pay much, so whatever data they could extract was ok for the cost.
The other point from article. I took this as experimental, so maybe we'll find out later that they really couldn't get much usable data.
I used to just record the ground and even leave out my feet, but apparently they detect and ban people who do this too often now. The data was always going somewhere shady, but after the sale it is even worse so I just stopped completely. At best you get a poffin or rare candy and that absolutely is not worth it.
Niantic are a number of people who are doing this. Its not that clear from the article, but niantic spatial are using the images captured from users to create a 3d model of "THE WORLD" or where people play pokemon go.
They have then fed that data into a more modern version of colmap (https://github.com/colmap/colmap) to create a point cloud. Then the engineering to make sure that point cloud is aligned accurately and automatically.
Once you have that point cloud aligned to the world, all you need is another image with some overlapping feature. Using simple trigonometry you can work out where the camera is from one picture
This is largely trivial to do for a few 100 sqaure meters. the hard part is doing it fast in at the city scale. Extracting a few thousand features from an image and then matching them against >billion other points is hard to do quickly, without some optimisations.
The thing that is not mentioned here is that data freshness is actually more important. Building change (advertising hoardings, paint jobs, logo changes, building remodelled etc) so the data goes stale. Its actually not that expensive anymore to just send your own people to scan areas. (A number of startups pre 2020 did it, mapillary provides a platform for it, although now owned by facebook)
The robots will be feeding that data back in to the map. the special sauce is updating the map without infringing patents, and doing it efficiently.
Niantic has been doing this for a long time starting with Ingress. I've maybe done a handful of scans in PoGo but as others have mentioned the rewards were just not worth the trouble. The rewards for doing it in Ingress were much better (at least back in the day).
I'm more split on my feelings towards it these days given our current political/social climate but part of me still thinks the idea of mapping the real world in great detail is a worthy endeavor if it can be done "right". I'd probably be more inclined to support it if they would release the data or make it publicly accessible for others to use but it being tied to the whims of a corporation (even one that's been less shitty than most) makes it hard to get behind.
At what point will we have people transmit their car dash cams along with GPS information in order to generate more data? I’m actually surprised this hasn’t happened yet with self driving car manufacturers needing more and more data
Thats because Tesla is useless, not because the data isn't valuable.
Tesla has explicitly ruled out using "HD maps" for autonomous vehicles. This means that all the data they have is going to not building maps, but building scenarios for testing its self driving models.
If you look at Wayve, they are building nerf maps to allow them to create scenarios for edge cases. all of that comes from the gathered data.
If you want to build visual navigation systems, you need lots of fresh data from all over. Seeding it with the data that naintic has is useful, but a lot of that data is out of date so not that useful anymore.
Great question. A "Ring Dashcam" with a mobile connection would win customers based on name recognition alone.
Not a lot of big companies in the dashcam market, there are a lot of alphabet companies and some small players like Vantrue. The only company with broader recognition is Garmin and it feels like a weird side gig for them.
It's the lack of transparency that is bad. PokemonGo did not make it clear it was taking (and uploading) pictures.
You could argue that "of course it must be for AR", but that isn't clear at all. The camera shows a live image before I take a photo, and I wouldn't expect a photo to be captured and sent if I didn't press the (virtual) shutter.
There are probably some cheap phones that do precisely that, and I'd be just as annoyed at them and raise the same concerns.
It isn't recording surreptitiously. The data was collected as part of an optional feature which is a very intentional process where you start a scan and then move around the object being scanned to get data from multiple angles, and then click to upload the data to Niantic. The uploading is called out specifically as a separate step (at least early on it was common for uploads to fail, so it had the option to save the scan to upload later when you had better signal). There is nothing secret about the fact that Niantic is collecting this data.
The lack of transparency is about how Niantic is using the data, selling it to third parties for purposes unrelated to the game. And I agree with the parent that this is a fair trade for a free game, especially since that part is optional, but more transparency would be better.
The article doesn’t say when this collection happens but there is some part of the game the involves photographing specific landmarks which does involve pressing a shutter. I’m guessing that’s where this comes from but would be great to hear from a better source.
It is not at all clear that the mapping is for purposes other than the AR features in the game itself though. In fact Niantic advertised the scanning field research as helping them make richer experience at PokeStops (which they did).
Niantic was much more upfront about this with Ingress, so people who know the company's history will likely guess that Pokemon Go is serving the same purpose, but for someone coming into the game without that background, there is nothing in the game itself that indicates that data is being collected for other commercial purpose.
Not unexpected, but it looks the oldest kid, Ingress, is being ignored again. IIRC, there was some badge you could earn by doing a number of those scans.
The OG was alive and kicking when I hopped back on a year and a half ago. No where near what it used to have as far as active players go but in my big city area there are still lots of active OGs and new people hopping on.
They were(are?) the same backend, same world maps, same POIs. Maybe they diversified at some point but at launch the Gyms were 1 to 1 with Ingress portals in my city.
It is interesting that the 'non-gaming' division of the split kept Ingress.
The question is how one stands on the monopolistic collection by a commercial entity.
I personally don't mind to share GPS traces and other data with i.e. open streetmap, as I directly benefit from the data as well and it's more or less equal between different entities.
I try not to give too much to Google and similar companies as this increases their competitive advantage, while my benefit is small.
Big corporations have found the way to make us work for free in their own terms. The balance of power between the working class and capital is totally broken.
And for me it is not just the lack of transparency. It is the power balance. I should not need to work for free, give my data, and god knows what to play a game. I should not be living knowing that I am being exploited at each interaction with software. Transparency is good, but not enough. "Click here to accept" and thousands of lines of legalese do not create a fair society.
You don't need to work and if you do you get rewarded in the game. Money is not the only motivating factor for people. Even something like keeping Google Map up to date can bring value to people from helping others. Helping others is not zero sum. Just because a company benefits from helping others that doesn't mean it's bad.
It’s the lack of transparency that is the problem. There should be a clear labor exchange disclaimer: “we are asking you to do X minutes of AI training for one unit of in-game reward.” What people take issue with is Tom Sawyer tricking people into whitewashing a fence.
You're right in that money is not the only motivator for people. I would also argue that if you told them the _real_ reason, aka your own actual motivation behind the offer, the number of people who would actually be "playing" would be much lower.
I would be motivated to collect free data if it meant I was helping save lives, with that help not being behind a paywall.
I would be motivated to play a free game with ads just for the fun of it.
I would not be motivated to play a free game just for the fun of it if my playing of the game was furthering some faceless corporation's profit motives.
In fact, in that last scenario, I would feel tricked, and it would take a non-trivial amount of money for me to not feel that way.
Same thing with social media. If they clearly disclosed that the more time they spend glued to their apps the more money the company makes the majority would be turned off.
The americans who believed anti-communist propaganda might have been the dumbest generation in history: they gave up a decent democracy with actual democratic say in how society is run in order to fear the world comfortably from home.
A long time ago they had the games that came with a "Pokéwalker", which was a pedometer that acted as a sort of mini-game, but that's the closest they got to a mainstream game where you have to get off the couch. It almost meets your specification (well, did, I'm sure the internet part is offline so you're not connected anymore), but it's obviously not exactly what you meant.
I didn't know, although I play Pokemon Go and the Wikipedia page for them doesn't say big data although it mentions "...been developing for years: the Niantic Real World Platform." as mentioned in the article although it sounds a bit of a work in progress. They make a lot of money as a games company.
Even without looking it up to know for sure it was pretty obvious and could be inferred by anyone playing the game. Especially the scanning, which was painfully obvious to be a data collection method.
I don't think anyone actually cares about this in principle. This is more of a product and marketing problem than a legal or moral one.
What people dislike is noticing the strings attached so distastefully. I can't think of any fads or pastimes where there aren't any, but the benefits of the activity offered should outweigh the cost.
In that sense, Pokémon Go was a bad deal. I still don't get what was ever in it for the player.
My friend plays Pokémon Go for hours every day while walking his dog. I asked him about this and now we’re both confused. The in game scanning is only for major landmarks in the game. Even in his dense city these landmarks are few and far between. The world model would only have sparse information in the area immediately surrounding these landmarks.
I don’t know if there’s much substance to the delivery robot story. This could be a journalist trying to make the story relatable.
I like Pokemon Go and play almost every day. I did this scan one time and then stopped. The rewards are not worth the hassle. I don't think many players are doing it. It's just very weird to stand somewhere and scan an object.
I also wouldn't say 'unknowingly trained', it's pretty obvious what it does, and I think the game even tells you that they want to understands how the POI looks like in 3D.
> I don't think many players are doing it.
Do we have to think? Apparently they amassed 30B images. :)
30B images isn't that much in the context of Pokemon Go playerbase of ~50 million (conservative estimate based on users today). That's about 600 images per person, in a game that has been out since 2016... that's pretty low adoption as the previous user said. I don't think the quest has been out since 2016, but considering a large fraction of users are basically daily users, it's still quite a small number of images.
That 30B could just be the useful set after cleanup.
My son showed me how he does these quests: he points the phone at the floor and just wiggles it back and forth.
Haha :D Good tip, I will try that too.
Your son is a bad data point.
Aww... shucks. I'm such a proud father.
Bad for some goals, good for others.
exactly. that is why i doubt they will get actual navigable information out of it.
To maintain that take, wouldn't you need to offer a plausible way that Niantic managed to train their Visual Positioning System using that data if the data was all bad?
I guess we don't know the terms of the deal, as far as how much they paid? So maybe they didn't pay much, so whatever data they could extract was ok for the cost.
The other point from article. I took this as experimental, so maybe we'll find out later that they really couldn't get much usable data.
I used to just record the ground and even leave out my feet, but apparently they detect and ban people who do this too often now. The data was always going somewhere shady, but after the sale it is even worse so I just stopped completely. At best you get a poffin or rare candy and that absolutely is not worth it.
Same, I do it once in a great while. Give me a rare candy or rare candy XL per scan and you’ll find me jumping all over the neighborhood!
Niantic are a number of people who are doing this. Its not that clear from the article, but niantic spatial are using the images captured from users to create a 3d model of "THE WORLD" or where people play pokemon go.
They have then fed that data into a more modern version of colmap (https://github.com/colmap/colmap) to create a point cloud. Then the engineering to make sure that point cloud is aligned accurately and automatically.
Once you have that point cloud aligned to the world, all you need is another image with some overlapping feature. Using simple trigonometry you can work out where the camera is from one picture
This is largely trivial to do for a few 100 sqaure meters. the hard part is doing it fast in at the city scale. Extracting a few thousand features from an image and then matching them against >billion other points is hard to do quickly, without some optimisations.
The thing that is not mentioned here is that data freshness is actually more important. Building change (advertising hoardings, paint jobs, logo changes, building remodelled etc) so the data goes stale. Its actually not that expensive anymore to just send your own people to scan areas. (A number of startups pre 2020 did it, mapillary provides a platform for it, although now owned by facebook)
The robots will be feeding that data back in to the map. the special sauce is updating the map without infringing patents, and doing it efficiently.
see this 2007 talk: https://www.ted.com/talks/blaise_aguera_y_arcas_how_photosyn...
i remember this being available in google maps in 2008 or so. fun technology!
I remember playing with Photosynth on my Windows Phone. Good times.
Niantic has been doing this for a long time starting with Ingress. I've maybe done a handful of scans in PoGo but as others have mentioned the rewards were just not worth the trouble. The rewards for doing it in Ingress were much better (at least back in the day).
I'm more split on my feelings towards it these days given our current political/social climate but part of me still thinks the idea of mapping the real world in great detail is a worthy endeavor if it can be done "right". I'd probably be more inclined to support it if they would release the data or make it publicly accessible for others to use but it being tied to the whims of a corporation (even one that's been less shitty than most) makes it hard to get behind.
> if they would release the data or make it publicly accessible for others
HAH! Deep dreams.
At what point will we have people transmit their car dash cams along with GPS information in order to generate more data? I’m actually surprised this hasn’t happened yet with self driving car manufacturers needing more and more data
https://www.getnexar.com/
Not obvious if they're successfully selling any of their collected data yet, but they must at least have plans to try.
Tesla does it and clearly it’s not all that useful in reality.
Thats because Tesla is useless, not because the data isn't valuable.
Tesla has explicitly ruled out using "HD maps" for autonomous vehicles. This means that all the data they have is going to not building maps, but building scenarios for testing its self driving models.
If you look at Wayve, they are building nerf maps to allow them to create scenarios for edge cases. all of that comes from the gathered data.
If you want to build visual navigation systems, you need lots of fresh data from all over. Seeding it with the data that naintic has is useful, but a lot of that data is out of date so not that useful anymore.
This.... absolutely already happens. It's trivial and cheap to purchase the three meter, three second resolution data of millions of vehicles.
Mobileye builds their maps like this
Great question. A "Ring Dashcam" with a mobile connection would win customers based on name recognition alone.
Not a lot of big companies in the dashcam market, there are a lot of alphabet companies and some small players like Vantrue. The only company with broader recognition is Garmin and it feels like a weird side gig for them.
Despite the lack of transparency, is this so bad? Players are being given a game in exchange for collectively building a database.
It's the lack of transparency that is bad. PokemonGo did not make it clear it was taking (and uploading) pictures.
You could argue that "of course it must be for AR", but that isn't clear at all. The camera shows a live image before I take a photo, and I wouldn't expect a photo to be captured and sent if I didn't press the (virtual) shutter.
There are probably some cheap phones that do precisely that, and I'd be just as annoyed at them and raise the same concerns.
It isn't recording surreptitiously. The data was collected as part of an optional feature which is a very intentional process where you start a scan and then move around the object being scanned to get data from multiple angles, and then click to upload the data to Niantic. The uploading is called out specifically as a separate step (at least early on it was common for uploads to fail, so it had the option to save the scan to upload later when you had better signal). There is nothing secret about the fact that Niantic is collecting this data.
The lack of transparency is about how Niantic is using the data, selling it to third parties for purposes unrelated to the game. And I agree with the parent that this is a fair trade for a free game, especially since that part is optional, but more transparency would be better.
I recall there being a pretty obvious notice when they first ask if you want to participate. Whether people read it is another thing.
The article doesn’t say when this collection happens but there is some part of the game the involves photographing specific landmarks which does involve pressing a shutter. I’m guessing that’s where this comes from but would be great to hear from a better source.
> crowdsourced data, seemingly collected for one purpose
> Whether players knew it or not, those scans were creating 3D models of the real world
Kind of shitty reporting. Did users know about this data collection or not? Was it not disclosed?
The privacy policy was just a generic corporate "we may collect some information to improve a service" crap.
Technically, lawyers will argue that users had to opportunity to inform themselves.
Practically, nobody knew.
The quests themselves are prominently labeled "AR mapping". You don't need to go into the privacy policy to know what they are.
It is not at all clear that the mapping is for purposes other than the AR features in the game itself though. In fact Niantic advertised the scanning field research as helping them make richer experience at PokeStops (which they did).
Niantic was much more upfront about this with Ingress, so people who know the company's history will likely guess that Pokemon Go is serving the same purpose, but for someone coming into the game without that background, there is nothing in the game itself that indicates that data is being collected for other commercial purpose.
And we’re all training ai right now with our comments. Even the bots here.
Not unexpected, but it looks the oldest kid, Ingress, is being ignored again. IIRC, there was some badge you could earn by doing a number of those scans.
Or is Ingress even still around?
The OG was alive and kicking when I hopped back on a year and a half ago. No where near what it used to have as far as active players go but in my big city area there are still lots of active OGs and new people hopping on.
They were(are?) the same backend, same world maps, same POIs. Maybe they diversified at some point but at launch the Gyms were 1 to 1 with Ingress portals in my city.
It is interesting that the 'non-gaming' division of the split kept Ingress.
Does anyone remember Ingress? I always wondered what it was we were training by playing that game.
Considering that Niantic was behind both ingress and pokemon go, the answer is all of this.
Millions of software devs unknowingly trained LLMs to start replacing their jobs.
PUSH your code to fit to find out more.
Looks like teenagers are going to have fun playing Pokemon Go, and now have faster food deliveries.
It's useful to map the world, this is what Google / Baidu / Yandex Maps are doing too.
It is indeed useful.
The question is how one stands on the monopolistic collection by a commercial entity.
I personally don't mind to share GPS traces and other data with i.e. open streetmap, as I directly benefit from the data as well and it's more or less equal between different entities.
I try not to give too much to Google and similar companies as this increases their competitive advantage, while my benefit is small.
Nintendo's brand means nothing.
Big corporations have found the way to make us work for free in their own terms. The balance of power between the working class and capital is totally broken.
And for me it is not just the lack of transparency. It is the power balance. I should not need to work for free, give my data, and god knows what to play a game. I should not be living knowing that I am being exploited at each interaction with software. Transparency is good, but not enough. "Click here to accept" and thousands of lines of legalese do not create a fair society.
You don't need to work and if you do you get rewarded in the game. Money is not the only motivating factor for people. Even something like keeping Google Map up to date can bring value to people from helping others. Helping others is not zero sum. Just because a company benefits from helping others that doesn't mean it's bad.
It’s the lack of transparency that is the problem. There should be a clear labor exchange disclaimer: “we are asking you to do X minutes of AI training for one unit of in-game reward.” What people take issue with is Tom Sawyer tricking people into whitewashing a fence.
You're right in that money is not the only motivator for people. I would also argue that if you told them the _real_ reason, aka your own actual motivation behind the offer, the number of people who would actually be "playing" would be much lower.
I would be motivated to collect free data if it meant I was helping save lives, with that help not being behind a paywall.
I would be motivated to play a free game with ads just for the fun of it.
I would not be motivated to play a free game just for the fun of it if my playing of the game was furthering some faceless corporation's profit motives.
In fact, in that last scenario, I would feel tricked, and it would take a non-trivial amount of money for me to not feel that way.
Same thing with social media. If they clearly disclosed that the more time they spend glued to their apps the more money the company makes the majority would be turned off.
The americans who believed anti-communist propaganda might have been the dumbest generation in history: they gave up a decent democracy with actual democratic say in how society is run in order to fear the world comfortably from home.
You reckon subsequent generations improved? Hint: look around and see what's happening, and how that came about.
For Maga, democracy is too socialist.
LOL: Remember when they literally said the Pope wasn't Christian enough.
Maga, all butt hurt with the slow, creeping realization, that they are the baddies.
Before: "Freeeeeeddommm of Speech".
Now: "News outlets must follow admin talking points or be hung for treason" We'll solve that buy just buying them out.
> I should not need to work for free, give my data, and god knows what to play a game.
The game is free, with doing so being the price. If you don't like that price, you can always pay $80 for a traditional Pokemon game.
As such, I don't get the handwringing. There is no such thing as a free lunch and never will be.
Is there an $80 Pokémon game where walking outside is a core mechanic and that facilitates interaction with a huge network of users?
A long time ago they had the games that came with a "Pokéwalker", which was a pedometer that acted as a sort of mini-game, but that's the closest they got to a mainstream game where you have to get off the couch. It almost meets your specification (well, did, I'm sure the internet part is offline so you're not connected anymore), but it's obviously not exactly what you meant.
Take a walk while you play your switch
"We've got pokemon go at home!"
wasnt this sort of obvious to anyone familiar with Niantic?
“What’s Niantic?”
looks back down at phone
I think that only the author of this post didn't know that. Everybody know that Niantic is a big data company.
I didn't know, although I play Pokemon Go and the Wikipedia page for them doesn't say big data although it mentions "...been developing for years: the Niantic Real World Platform." as mentioned in the article although it sounds a bit of a work in progress. They make a lot of money as a games company.
Everyone in a very specific bubble
Even without looking it up to know for sure it was pretty obvious and could be inferred by anyone playing the game. Especially the scanning, which was painfully obvious to be a data collection method.
Pizza delivery robot, or a live grenade delivery robot, depending on the country and the dataset buyer.
I don't think this is the worst thing trained.
Niantic builds massive geospatial models that can localize and reconstruct views: https://www.nianticspatial.com/
Extremely detailed mappings of CONUS with spatial intelligence already built around it, and we let the company get sold to Saudi government last year.
Pretty sure you take a video when you’re doing this task so the images are extra extracted from the video?
I'm not positive this was a secret (See: Reddit post about it from 2018):
https://www.reddit.com/r/TheSilphRoad/comments/8i7byi/pokemo...
Stalking as a service - upload a photo of your target and get back gps coordinates, coming soon!
Let's turn it into a gig app.
PIs need to live in the future too.
Swipe left/right if you want to take the job.
I don't think anyone actually cares about this in principle. This is more of a product and marketing problem than a legal or moral one.
What people dislike is noticing the strings attached so distastefully. I can't think of any fads or pastimes where there aren't any, but the benefits of the activity offered should outweigh the cost.
In that sense, Pokémon Go was a bad deal. I still don't get what was ever in it for the player.
Those pokestop scans are trash. I can't believe they will get enough detailed information out of them to allow navigation by robots.
I knew, or strongly assumed, and think it’s a great dynamic.
> The massive crowdsourcing effort could use real-world to help robots deliver pizza.
Huh? Does popsci not have copy editors?
My friend plays Pokémon Go for hours every day while walking his dog. I asked him about this and now we’re both confused. The in game scanning is only for major landmarks in the game. Even in his dense city these landmarks are few and far between. The world model would only have sparse information in the area immediately surrounding these landmarks.
I don’t know if there’s much substance to the delivery robot story. This could be a journalist trying to make the story relatable.
I was thinking the 'scan a pokestop' data would be very patchy but I guess they also have images from people catching pokemon in AR mode?