Meta gains the right to crawl company websites. Who does this affect?
From October 30, every company that runs a Facebook page, advertises, or sells through Meta’s tools will agree to let the company copy content from its website and use it also to train AI. There is no opt-out option.
Meta has published new Commercial Terms, that is, rules for everyone who uses its products for business purposes. The company itself presents the change as streamlining documents and reducing repetition. Lawyers, trade newsrooms, and publisher organizations read it differently. In their view, consent to automatic content collection has so far applied to advertisers, and now it will cover virtually every company present on Meta’s platforms.
The issue was revealed on October 15 by Nieman Lab, and was then analyzed in detail by Search Engine Journal and Ars Technica. We checked who the new provisions apply to, what the company is agreeing to, and what it can do in the less than two weeks left before the changes take effect.
The new terms will cover every company on Facebook, not just advertisers
The new Commercial Terms apply to any business use of Meta’s products. These include running ads, a company Facebook page (Page), marketing messages, product catalogs, and the Ads Manager and Meta Business Suite tools. You do not need to buy a single ad for the terms to start applying. The same clause has also been added to the Self-Serve Ad Terms, which also take effect on October 30.
The current Commercial Terms, dated March 6, 2024, do not mention crawling at all, that is, the automatic browsing and copying of websites by programs called crawlers. Until now, such authorization existed in the advertising terms and in the Seller Agreement. However, it applied to the advertiser’s domains or addresses that the advertiser or seller had provided themselves. It was also tied to a specific action, such as ordering an ad, preparing its content, or updating a catalog. The new version covers all domains and addresses “associated with” the company’s products or services, as well as those shared by the company or by users of Meta’s products.
What a company agrees to by accepting the terms
The clause authorizes Meta, as well as any entity acting on its behalf, to use scrapers (programs that automatically copy website content), bots, and crawlers. It allows them to index websites, store copies of them, analyze them, and collect their content, as well as periodically retrieve updates. Meta may use the collected content to provide, develop, improve, research, train, and maintain the integrity of its products. Ars Technica notes that the word “AI” does not appear in the clause even once. The training of models is indicated only by the word “train” in combination with automatic data collection.
At the same time, it is not possible to use Meta’s tools and refuse this consent. Under the terms, continued use of Meta’s services for business purposes means acceptance of the new rules. A company that does not agree must stop using Meta’s products for business purposes.
The provision causing the most questions is the one allowing Meta to create “test credentials,” meaning login details for test accounts. According to Ars Technica, it had previously appeared only in the Seller Agreement, where it had a narrow purpose: helping optimize offers. James Grimmelmann, a law professor at Cornell Law School, believes this could technically allow Meta to register as a fictitious user and bypass a registration gate. Meta declined to answer whether it could in this way bypass paywalls, that is, paid barriers to content access.
Meta speaks of streamlining, lawyers of expanded powers
Meta spokesperson Andy Stone on October 16 called the summary of the Nieman Lab report false on platform X. He wrote that the change is intended to allow the company to obtain accurate information about products sold on external websites when Meta recommends ads and content. He added that this is not about scraping news sites. The Meta spokesperson also told Ars Technica that the way crawlers are used is not changing. Meta also points out that content owners can still control access to their content through the robots.txt file, that is, instructions for bots placed on the website server.
The problem is that neither the new Commercial Terms nor the new Self-Serve Ad Terms mention robots.txt. Lawyers cited by Nieman Lab believe this gives Meta grounds to argue in the future that the contract takes precedence over blocks set on the website. Matt Topic of the law firm Loevy & Loevy, who represents The Intercept in its copyright dispute with OpenAI, admits that it is unclear whether a court would consider this provision binding. Publishers should, however, assume that it will, in his view.
It is extraordinary that such a provision was hidden in the terms, since it would give Meta access to materials that would probably cost many millions of dollars on the licensing market.
This dispute cannot be resolved today. Meta assures that in practice nothing is changing, but the text of the terms gives it broader powers than before.
What a company can lose: control over content and licensing revenue
Content has a tangible price for companies building AI. In December 2025, Meta announced its first licensing deals with publishers, including CNN, Fox News, and USA Today. According to industry reports, in March 2026 it also signed a deal with News Corp worth up to $50 million (approx. PLN 180 million) per year for three years. Critics point out that the new clause could give Meta access to similar materials from other companies without any payment. Grimmelmann adds that Meta could use this content on its platforms even in competition with its authors. None of the sources explains how the new terms relate to publishers that already have licensing agreements with Meta.
The issue does not concern only the media. Danielle Coffey, head of the U.S. publisher organization News/Media Alliance, called the new terms “outrageous and probably illegal” (trans. ed.). She warns that the provision is so broad that it threatens every company that publishes its own content online and has a business account with Meta. In Europe, the European Publishers Council, an association of newspaper publishers, has turned to the European Commission. The organization is concerned that Meta could treat a user’s sharing of a link to an article as consent to use that article.
On the other side of the balance sheet is what companies get in return: reach, website traffic, and advertising tools. For many of them, it is hard to find a substitute. This is clearly visible in the case of Stephen Beale, creator of the niche site The Steampunk Explorer. Although his site lives off traffic from Facebook, Beale is considering leaving the platform and tells Nieman Lab that he does not want Meta to “get away with it.”
What to do before October 30
Given the current state of knowledge, companies have three options. They can accept the terms, restrict crawler access, or stop using Meta’s business products. Regardless of the choice, it is worth doing a few things:
- Check which Meta tools the company uses: Facebook pages, Business Suite, the product catalog, or ads. Each of them means acceptance of the new terms.
- Review the robots.txt file for each Meta crawler.
- Consider whether the website contains login-protected or paid content that requires additional protection.
- For content with high licensing value, consult a lawyer about the effects of the new terms.
Search Engine Journal, citing Meta’s documentation, indicates that it is possible to block the crawler that collects data for model training (Meta-ExternalAgent). At the same time, access can be left open for Meta-WebIndexer, which according to Meta helps quote and link content in Meta AI responses. The block looks like this:
User-agent: meta-externalagent
Disallow: /This solution has its limitations. According to Meta’s documentation, Meta-ExternalFetcher may ignore robots.txt when it retrieves a page at a user’s request. FacebookExternalHit, which creates link previews, may do the same during security checks. The new terms also do not say whether robots.txt settings limit the scope of the consent a company gives by accepting the agreement.
In our view, for most companies leaving Facebook is not a realistic option. However, they should not treat this change like another terms update that gets clicked through without reading. The minimum is a conscious decision, a properly configured robots.txt, and watching whether the European Commission responds to the matter.