SOLVED: WordPress 404 THE REQUESTED URL WAS NOT FOUND ON THIS SERVER

We recently worked through a particularly difficult to locate problem in which periodically some visitors would see the following 404 THE REQUESTED URL /commodore.guest-bike.php WAS NOT FOUND ON THIS SERVER

The problem very intermittent although we were able to see the issue ourselves from time to time.  It turned out that a hack we had cleared a year or so ago was still affecting the site.  In this hack some visitors were being redirected and we used several tools including Wordfence to clear the problem.  When Wordfence and some third party tools validated that the site was clean, we moved on.

However, it turned out that there were some residual changes which Wordfence and others were not inspecting and that was causing us grief.  In particular the good people in GoDaddy’s advanced hosting support found the following at the top of our .HTACCESS file:

RewriteCond %{HTTP_USER_AGENT} (google|yahoo|msn|aol|bing) [OR]
RewriteCond %{HTTP_REFERER} (google|yahoo|msn|aol|bing)
RewriteRule ^(.*)$ guest-bike.php?$1 [L]

As you can see this redirected visitors from Google, Bing, Yahoo and others to guest-bike.php which was an infected file that Wordfence had cleared off.

The solution was to edit the .HTACCESS file using any text editor (like Notepad or Notepad ++) and remove those redirects.

.HTACCESS is the Linux Apache web servers equivalent of Windows web servers WEB.CONFIG and it is what tells the web server software which features to enable or disable and other low level functions.  You can access .HTACCESS using FileZilla or your hosts (i.e. GoDaddy) file management tools.  For more information on .HTACCESS see www.htaccess-guide.com.

After we had HTACCESS cleaned up we wanted to the Google to reindex our site:

1. surf to the Google Search Console for administrators at search.google.com/search-console
2. click URL INSPECTION in the left hand menu
3. Enter this site in question
4. Click REQUEST INDEXING on the top panel