User-agent: Scrapy
Allow: /

User-Agent: *
Allow: /
Disallow: /artikel/
Disallow: /clickheat/
Disallow: /css/
Disallow: /gfx/
Disallow: /images/
Disallow: /inc/
Disallow: /neu/
Disallow: /v2/
Disallow: /wdw_suche/
Disallow: /weg-mit-der-panik/