User-agent: * # 1. 核心系统与隐私目录 (允许JS/CSS渲染,保护日志与缓存) Disallow: /wp-admin/ Disallow: /wp-content/cache/ Disallow: /wp-content/uploads/wc-logs/ Disallow: /wp-content/uploads/woocommerce_transient_files/ Disallow: /wp-content/uploads/woocommerce_uploads/ Disallow: /author/ Disallow: /xmlrpc.php # 2. 用户中心与交易页面 (保护隐私,防止收录无效/重复页面) Disallow: /cart/ Disallow: /checkout/ Disallow: /my-account/ Disallow: /order/ Disallow: /order-received/ Disallow: /lost-password/ # 3. 动态参数与系统冗余过滤 (防止WooCommerce筛选器产生垃圾URL) Disallow: /*?add-to-cart= Disallow: /*?*add-to-cart= Disallow: /*?attribute_pa_ # 修复:必须用 /*? 拦截全站分类下的属性筛选 Disallow: /*?filter_ Disallow: /*?orderby= Disallow: /*?add-to-wishlist= Disallow: /*?s= Disallow: /search/ # 新增:屏蔽站内搜索结果页,防止低质页面被收录 Disallow: /*?replytocom= Disallow: /*?share= Disallow: /*?min_price= Disallow: /*?max_price= Disallow: /*?wc-ajax= Disallow: /*/feed/ # 新增:屏蔽RSS源,节省抓取预算 Disallow: /*/trackback/ # 新增:屏蔽Trackback,防垃圾外链通知 # 4. 允许核心 AJAX 与媒体资源 (确保Cosplay精美图片被收录) Allow: /wp-admin/admin-ajax.php Allow: /wp-content/uploads/ # ========================================== # 显式放行:社交媒体预览爬虫 (确保分享时显示精美封面图) # ========================================== User-agent: facebookexternalhit Allow: / User-agent: Facebookbot Allow: / User-agent: Twitterbot Allow: / User-agent: Pinterestbot Allow: / User-agent: LinkedInBot Allow: / User-agent: WhatsApp Allow: / User-agent: TelegramBot Allow: / User-agent: Discordbot Allow: / # ========================================== # 显式放行:主流搜索引擎 (确保SEO自然流量正常收录) # ========================================== User-agent: Googlebot Allow: / User-agent: Googlebot-Image Allow: / User-agent: Bingbot Allow: / User-agent: Baiduspider Allow: / User-agent: YandexBot Allow: / User-agent: DuckDuckBot Allow: / # ========================================== # 严格屏蔽:AI 纯训练爬虫 (保留最小屏蔽面; GPTBot/ChatGPT-User/PerplexityBot 已于 2026-08-28 放行) # ========================================== User-agent: ClaudeBot Disallow: / User-agent: Claude-Web Disallow: / User-agent: Claude-User Disallow: / User-agent: anthropic-ai Disallow: / User-agent: Bytespider Disallow: / User-agent: CCBot Disallow: / User-agent: Applebot-Extended Disallow: / User-agent: Google-Extended Disallow: / User-agent: Omgilibot Disallow: / User-agent: Omgili Disallow: / User-agent: Diffbot Disallow: / User-agent: YouBot Disallow: / User-agent: ImagesiftBot Disallow: / User-agent: Meta-ExternalAgent Disallow: / User-agent: Meta-ExternalFetcher Disallow: / User-agent: Amazonbot Disallow: / # ========================================== # 严格屏蔽:第三方 SEO 分析工具与高耗能爬虫 (无实际转化,纯消耗性能) # ========================================== User-agent: AhrefsBot Disallow: / User-agent: AhrefsSiteAudit Disallow: / User-agent: SemrushBot Disallow: / User-agent: SemrushBot-BA Disallow: / User-agent: MJ12bot Disallow: / User-agent: DotBot Disallow: / User-agent: DataForSeoBot Disallow: / User-agent: BLEXBot Disallow: / User-agent: SeekportBot Disallow: / User-agent: Rogerbot Disallow: / User-agent: PetalBot Disallow: / Sitemap: https://ed2kcosplay.com/sitemap_index.xml # ========================================== # AI 引用/搜索增强爬虫:放行 (2026-08-28 批准 — 带来可转化流量) # ========================================== User-agent: GPTBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: PerplexityBot Allow: /