Beschreibung
AI Scrape Protect is a WordPress plugin designed to protect your website from scraping for AI training purposes. It achieves this by adding opt-out instructions to the robots.txt file for the most common AI scraping bots and including meta tags to control how your content is used.
Note: These instructions are not always respected by all bots.
Features
- Adds specific
User-agentandDisallowrules to yourrobots.txtfile to block a comprehensive list of AI scraping bots. - Introduces meta tags in the HTML
<head>to provide additional instructions to AI bots, including new tags for Bingbot and general AI compliance. - Prepares for future standards with support for
DisallowAITrainingandnoimageaimeta tags. - Dedicated handling of specific bots like CCBot and Bingbot for better protection and compatibility.
- Admin bar icon to indicate plugin activity.
Screenshots
robots.txt File Example: Shows how the plugin updates the robots.txtfile.
meta tags Example: Shows an example of the Meta Tags added to the head section.
Installation
- Upload the
ai-scrape-protectfolder to the/wp-content/plugins/directory. - Activate the plugin through the ‚Plugins‘ menu in WordPress.
FAQ
-
How does this plugin protect my site from AI scraping?
-
The plugin adds specific
User-agententries to yourrobots.txtfile to instruct common AI scraping bots not to crawl or scrape your site. It also introduces meta tags in the HTML<head>to provide additional instructions to AI bots. -
Will this completely stop AI scraping of my site?
-
While this plugin adds recommendations to the
robots.txtfile and includes meta tags, not all bots follow these rules. This is a measure to discourage scraping rather than a foolproof solution. -
Can I add or remove bots from the list?
-
Currently, the plugin includes a predefined list of bots. If you need to add or remove specific bots, you would need to modify the plugin code or contact the plugin author for customization.
-
What happens if I deactivate the plugin?
-
The
robots.txtfile will revert to its previous state before the plugin was activated, and the meta tags added to the HTML<head>will be removed.
Rezensionen
Mitwirkende & Entwickler
„AI Scrape Protect“ ist Open-Source-Software. Folgende Menschen haben an diesem Plugin mitgewirkt:
Mitwirkende
Übersetze „AI Scrape Protect“ in deine Sprache.
Interessiert an der Entwicklung?
Durchstöbere den Code, sieh dir das SVN Repository an oder abonniere das Entwicklungsprotokoll per RSS.
Änderungsprotokoll
3.1
- Added new AI bots to the block list: DuckDuckBot, OpenAIContentCrawler, YandexBot, NeevaBot, AIMatrixCrawler.
- Improved admin bar icon clarity with higher resolution and visual indicators (green dot for active state).
3.0
- Added admin bar icon functionality to indicate plugin activity.
- Updated meta tags for compliance with official documentation and improved AI scraping protection:
- Adjusted Bingbot tag to use
nocachefor better compatibility with Bing AI Chat. - Removed „noindex“ from the
ai-botmeta tag to allow search engine indexing. - Added
DisallowAITrainingandnoimageaito therobotsmeta tag. - Introduced dedicated meta tags for CCBot.
- Adjusted Bingbot tag to use
- Removed OpenAI SearchBot from the
robots.txtblocklist based on OpenAI’s recommendations.
2.4
- Updated meta tags for improved AI scraping protection.
- Prevent Bingbot and general AI bots from using the content for AI purposes.
- Added clear comments in the code to describe the functionality of each meta tag.