seo

Robots.txt

File telling crawlers what to access.

Definition

robots.txt is a text file at your site root that tells search engine crawlers which pages they can or cannot crawl. Not a security measure — just a politeness request.

Related terms

SEO

Search engine optimization.

SERP

Search engine results page.

Keyword

A search query target.

Long-tail keyword

Specific, low-volume search queries.

Put Robots.txt to work

Build, publish, and rank your content with RankFlo. Free to start.

Start for free