Topic: Admin and Users page indexed by Google!

I´ve created a Monstra Website. But, when indexed by Google, I was scared that these pages were indexed:
/users
/users/registration
/uservs/login
/admin
/users/password-reset

How to make these pages "noindex"?

Re: Admin and Users page indexed by Google!

They are indexed cause google cant read robot.txt which prevent google indexing those places, u have to download monstra 3.0.3 again and take .htsacces file and replace with yours if u got monstra 3.0.3

(с) Roman Art
So far So good wink

Re: Admin and Users page indexed by Google!

Hi RomanArt, thanx for your gently and fast contribution.

I edited the "robots.txt" and included "/users" folder (my Monstra version is 3.0.3).

But then I realized that the problem is not "robots.txt", is .htaccess. Because I tryed to access that file and it returned 404 Monstra page.

Every file I put in the root is unreacheable if it is not a Monstra generated page.

I try "mydomain.com/robots.txt" and it returns a Monstra 404 page.

I´ve uploaded an "abc.html" to root and tried to access from browser and it returns Monstra 404 page.

Here is the .htaccess created by Monstra. How can I fix it?

==============================================

#
# This file is part of the Monstra.
#
# (c) Romanenko Sergey / Awilum <awilum@msn.com>
#
# For the full copyright and license information, please view the LICENSE
# file that was distributed with this source code.
#

# Set default charset utf-8
AddDefaultCharset UTF-8

# PHP 5, Apache 1 and 2.
<IfModule mod_php5.c>
    php_flag magic_quotes_gpc                 off
    php_flag magic_quotes_sybase              off
    php_flag register_globals                 off
</IfModule>

<IfModule mod_rewrite.c>
    RewriteEngine on

    ## Begin - Rewrite rules to block out some common exploits.
    # If you experience problems on your site block out the operations listed below
    # This attempts to block the most common type of exploit `attempts` to Monstra
    #
    # Block out any script trying to base64_encode data within the URL.
    RewriteCond %{QUERY_STRING} base64_encode[^(]*\([^)]*\) [OR]
    # Block out any script that includes a <script> tag in URL.
    RewriteCond %{QUERY_STRING} (<|%3C)([^s]*s)+cript.*(>|%3E) [NC,OR]
    # Block out any script trying to set a PHP GLOBALS variable via URL.
    RewriteCond %{QUERY_STRING} GLOBALS(=|\[|\%[0-9A-Z]{0,2}) [OR]
    # Block out any script trying to modify a _REQUEST variable via URL.
    RewriteCond %{QUERY_STRING} _REQUEST(=|\[|\%[0-9A-Z]{0,2})
    # Return 403 Forbidden header and show the content of the root homepage
    RewriteRule .* index.php [F]
    #
    ## End - Rewrite rules to block out some common exploits.

    ## Begin - Rewrite rules for Monstra
    RewriteBase /
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteCond %{REQUEST_FILENAME} !-d
    RewriteRule ^(.*)$ index.php [QSA,L]
    ## End - Rewrite rules for Monstra

    ## Begin - Rewrite rules for SEO improvements.
    # RewriteCond %{HTTP_HOST} ^www.example.org [NC]
    # RewriteRule ^(.*)$ http://example.org/$1 [R=301,L]
    # Redirect 301 /index http://example.org/
    ## End - Rewrite rules for SEO improvements.

</IfModule>

# Prevent visitors from viewing files directly.
<FilesMatch "(^#.*#|\.(md|txt|html|tpl|yml|yaml)|~)$">
    Order allow,deny
    Deny from all
    Satisfy All
</FilesMatch>

# Don't show directory listings for URLs which map to a directory.
Options -Indexes

#Force non-www:
RewriteCond %{HTTP_HOST} ^www\.mydomain.com\.net [NC]
RewriteRule ^(.*)$ http://mydomain.com/$1 [L,R=301]

#Compressao Gzip
<IfModule mod_deflate.c>
  # Compress HTML, CSS, JavaScript, Text, XML and fonts
  AddOutputFilterByType DEFLATE application/javascript
  AddOutputFilterByType DEFLATE application/rss+xml
  AddOutputFilterByType DEFLATE application/vnd.ms-fontobject
  AddOutputFilterByType DEFLATE application/x-font
  AddOutputFilterByType DEFLATE application/x-font-opentype
  AddOutputFilterByType DEFLATE application/x-font-otf
  AddOutputFilterByType DEFLATE application/x-font-truetype
  AddOutputFilterByType DEFLATE application/x-font-ttf
  AddOutputFilterByType DEFLATE application/x-javascript
  AddOutputFilterByType DEFLATE application/xhtml+xml
  AddOutputFilterByType DEFLATE application/xml
  AddOutputFilterByType DEFLATE font/opentype
  AddOutputFilterByType DEFLATE font/otf
  AddOutputFilterByType DEFLATE font/ttf
  AddOutputFilterByType DEFLATE image/svg+xml
  AddOutputFilterByType DEFLATE image/x-icon
  AddOutputFilterByType DEFLATE text/css
  AddOutputFilterByType DEFLATE text/html
  AddOutputFilterByType DEFLATE text/javascript
  AddOutputFilterByType DEFLATE text/plain
  AddOutputFilterByType DEFLATE text/xml

  # Remove browser bugs (only needed for really old browsers)
  BrowserMatch ^Mozilla/4 gzip-only-text/html
  BrowserMatch ^Mozilla/4\.0[678] no-gzip
  BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
  Header append Vary User-Agent
</IfModule>

#Cache do Browser
## EXPIRES CACHING ##
<IfModule mod_expires.c>
ExpiresActive On
ExpiresByType image/jpg "access plus 1 year"
ExpiresByType image/jpeg "access plus 1 year"
ExpiresByType image/gif "access plus 1 year"
ExpiresByType image/png "access plus 1 year"
ExpiresByType text/css "access plus 1 month"
ExpiresByType application/pdf "access plus 1 month"
ExpiresByType text/x-javascript "access plus 1 month"
ExpiresByType application/x-shockwave-flash "access plus 1 month"
ExpiresByType image/x-icon "access plus 1 year"
ExpiresDefault "access plus 2 days"
</IfModule>
## EXPIRES CACHING ##

Re: Admin and Users page indexed by Google!

I´ve found the problem in Monstra .htaccess:
I´ve just erased "txt" and "html" from the code below and now google can access robots.txt and analytics can access the .html they give you to check your property... Hope Monstra fix this in following versions...


# Prevent visitors from viewing files directly.
<FilesMatch "(^#.*#|\.(md|txt|html|tpl|yml|yaml)|~)$">
    Order allow,deny
    Deny from all
    Satisfy All
</FilesMatch>

Re: Admin and Users page indexed by Google!

This "problem" with html cause problem with ckeditor also.