#Screaming frog seo spider authentication install
The regular expressions you’ll need to use for a default WordPress install will look something like this: We want to exclude the URL that logs us out, and it’s also a good idea to put a blanket ban on crawling anything in /wp-admin/. It’s also a good idea to blacklist a few choice URLs using the SEO Spider’s Exclude feature. You may need to speak to your website’s administrator to get a read-only account for your development site set up. Our test site used the My Private Site WordPress plugin to password protect the entire site, which restricts access to logged in users only. During testing we created a new user just for the Spider with its role set to ‘subscriber’. The best and safest way to stop the SEO Spider from causing damage to your site is to ensure that you log it in with an account that doesn’t have write permissions on the website. The SEO Spider clicks every link on a page when you’re logged in that may include links to log you out, create posts, install plugins, or even delete data. This is a very powerful feature, and should therefore be used with great responsibility. The SEO Spider already supported standards based authentication (basic and digest authentication), but web forms authentication allows it to crawl virtually anything behind a login. In version 7.0 of the SEO Spider we released web forms authentication, which makes it possible to crawl development versions of websites or pages which have logins built into the page, such as a private WordPress site.
0 kommentar(er)
