tcggetBETA

tcggetBot crawler policy

User-Agent: tcggetBot/1.0 (+https://tcgget.com/about-bot)

What this bot does

tcgget collects and displays Pokémon card release, pre-order and lottery dates and prices across Korea, Japan and the US. This bot gathers only the titles, links and dates of public announcements and video listings. We never copy or republish full text, and every item we collect links back to the source.

What we honour

  • We check robots.txt first. For disallowed paths we never issue the request at all.
  • If robots.txt is unreachable — If we cannot read it, we do not crawl. a 404 means no restrictions, but a timeout or a 5xx means «unknown», not «allowed».
  • We honour Crawl-delay. Where none is given we wait a conservative 2 seconds.
  • Where an official RSS feed or API exists, we always use it. HTML crawling is a last resort.
  • We crawl each source three times a day. We do not fetch on every visitor request; we batch it.
  • We do not collect anything behind a login or paywall, and we do not circumvent access controls.

Sources we crawl automatically (8)

Sources we deliberately do not crawl (5)

  • PokéBeachrobots.txt permits it, but the server returns 403 to an identified bot user agent — feed, sitemap and home page alike. Getting around that would mean pretending to be a browser, which is circumventing a technical access control, so we do not.
  • Nintendo Liferobots.txt itself returns 403. Following our rule that we do not crawl what we cannot read the rules for, this source is excluded.
  • Serebii.netrobots.txt permits it, but there is no RSS and every link on the home page is Pokédex navigation, so a general extractor cannot pull articles. A site-specific parser would silently start returning nothing the moment the markup changed, so we do not write one.
  • Naver Cafe (Pokémon TCG)The content sits behind a login, so automated collection would breach the terms of service. Only public posts are checked by a person.
  • KREAM Trading CardsThere is no public API and the terms restrict automated collection. Prices are taken from the public ranking page by a person.

If you would rather we did not

The surest way is to add the following to your robots.txt. It takes effect from our next run.

User-agent: tcggetBot
Disallow: /

You can also just email us. We will remove you from the list on verification and reply — contact@tcgget.com